What’s next for smart contract transparency with Sourcify

M. Conor Svensson: If we could start with each of you just talking about who you are and what it is you do. So I'll just go in the order I can see people's names here. So Kaan first, then Marco and then Manuel.

Kaan: Yeah. Hi everyone, thanks for the space. so my name is Kaan. have been working on Sourcify for three years. We're three people. I'm one of the software developers and based in Berlin. Yeah, that's all from my side for now.

M. Conor Svensson: Marco.

Marco: Yes. So I'm Marco, based in Italy and I'm working on Sourcify, I think starting two years ago. And I yeah, working on decentralizing and making verification open source.

Mana: Yeah, hello, I'm Mana, I'm the new member of the team I just joined in April. Yeah I'm also based in Berlin, working in the same office, Kaan and me and yeah that's it I think. 

Conor Svensson: Thanks everyone for your intros. And it's really cool to have well, the whole team here for the space as well. I thought we might only get one of you, but to have all three of you is brilliant. And so let's just start with the origin of Sourcify. I don't know who wants to start with answering this question, but how was it that Sourcify came and came to be?

Kaan: Maybe I'll go with this because I think I'm the oldest team member, let's say longest standing. So I joined, as I said, I joined source five years ago, and at the time, actually 2021, September, exactly three years ago. At the time I joined the solidity team under the Ethereum Foundation, EF, and also Sourcify, still an EF project. I joined the solidity team and at the time Sourcify was a side project of solidity. So the main, let's say, goal and idea is one. There are no open source verification services like the current de facto standard in the ecosystem is a closed source company to solidity. Solidity has a feature called contract metadata. I think we'll get into details later on in the spaces. But the goal of certify is also to foster the use of solidity contract metadata. So it started as a smaller MVP. It was built by Shard Labs in Croatia. And then when I joined the project was brought back inside the EF and inside the Solidity team. And from there, we kind of kick started. And over the years, like I think two years ago or one, two years ago, we spun out as our own team inside EF. So now we are a separate team of three people.

M. Conor Svensson: Awesome. Thanks, Kaan. And so let's just unpack a little bit more on the things that you mentioned there as well. Source code verification. Why is it such an important topic for Ethereum?

Kaan: I mean, source code verification is the basis of everything, I would say everything. All the premises of blockchain, starting from transparency right in front of you, decentralization. I think these are all the kinds of things. Without source code verification I would say not much works in the blockchain space. Do you guys want to add anything? Marco, Emanual?

Marco: No, no no. I think you said enough.

M. Conor Svensson: Cool. And so in terms of the Sourcify service, it's become quite an important part of the Ethereum ecosystem with integrations with many of the different libraries that are used for developing and the deployment of smart contracts. Where is Sourcify headed? What are some of the big challenges for you? Because, you know, there was always this key issue there, which is about verification, and that the metadata, as you alluded to earlier, is provided in solidity contracts when they're deployed onto the network. But with problems when you find a solution such as contract verification, it's always more nuanced than people imagine it should be. You know, where kind of are we right now and where are we sort of headed, with respect to Sourcify?

Kaan: It's funny because we just did our quarterly planning and we kind of had an overview for the next couple quarters, maybe. Do you guys want to go first, Marco, Manuel. And then maybe I'll add some stuff?

Marco: Yeah. Okay. So, I think, that the main problem for us to be used more, let's say, was at the beginning, like just being integrated into all the tools that developers are using, while building web three applications. But I think that, yeah, like being part of the Ethereum foundation, being close to solidity helped us a lot to build and get seen by like for example Hard dot or Remix, and other frameworks. So I think that from that point of view we did great in the last years. but then at some point we started to struggle with optimization processes. So we are receiving a lot of contracts every second. And so for every contract there is a compilation going on. So I think probably this was the big achievement of the last month, I'd say yeah. And I think that now we are ready to start focusing on the big next steps that could be for example other languages integration. And then we have a, we want to maintain a big focus on decentralizing source code verification. So for example we want to make Sourcify run directly on the people's device. So this will be another big topic for the next step, let's say. And then the other big topic is to create these, I don't want to spoil too much, but create these let's say federated open databases of all of the verified contracts and these being worked on with the Verifier Alliance. So yeah, I think I like this roadmap of ours because it is close to our values. That is like making open source, making verification more open source and more decentralized.

M. Conor Svensson: That's awesome to hear. And certainly there must be some fascinating metrics that the team has seen then about the verification service because as you said to start with the challenge was getting more awareness of the service. And then you started getting all of these integrations with various pieces of developer tooling. But then to hit a point where the service is needing to scale with the actual demand of it, which obviously shows you're solving while providing a service that people really value. Are you able to provide some insights about what you've seen in terms of maybe metrics or other things about developer activity within the ethereum ecosystem on solidity via this? Because that idea of scaling to the point where you're having to do verifications every second, speaks obviously to the significant size of the community developing on top of the EVM. But are there any metrics or numbers that kind of come to mind which can help people appreciate quite how big all of this ecosystem is now?

Kaan: yeah, I think one recent metric might be from the roll ups. I can confidently say right now most of the stuff is happening on the roll ups and also most of our contracts are in Optimism and Base. So we see a lot of activity there. And also in terms of numbers we have something around I think one half or one verification per second right now, like a successful verified contract every second or so, which is, which can be a big number. But I think we are still in the beginning. As Marco said, we kind of integrated into tools, but I believe we still have a long way to go because still the de facto standard is Etherscan and that's ultimately what we want to change. And it turns out it's quite difficult to change people's behavior right now. One way we try to get around this is to try to get parallel verification, what I want to call parallel verification by default in tooling. For example, we built a remix plugin and we will release it soon. And there, for example, the default will be to verify on multiple verifiers instead of just going to Etherscan. So we are trying to push the ecosystem in this direction. If you want to verify, just verify everywhere. Why are we just verifying on Etherscan? And the change of this changing people's habit is the most difficult part, I would say.

M. Conor Svensson: Yes, it's a funny point that you mentioned there just around the changing of habits because of the amount of conversations we've had with people. It's kind of, they talk about block explorers, for instance, and they want the Etherscan experience. There's very much so. It's amazing how strongly these incumbent tools, so to speak, can actually frame people's thinking rather than step away from it there and think what's actually best for users generally. So it's a very key point, point there. So you sort of said how there's this idea of parallel verification, but then there's also the notion of decentralizing the process. Now maybe we're kind of getting ahead of ourselves because I know that as you say, this is something that you really want to be able to achieve longer term, but is it something at this point in time the team's spending much time thinking about, or is it more at this point? It's a long term goal, but there's more pressing issues that need to be addressed first.

Kaan: I would say it's a bit further in the roadmap still. So mainly what we are referring to here is to make Sourcify runnable on browsers so that you don't even need us to verify the whole thing. It's still a bit down the road. It's still not immediate because there are other more important problems. I would say here, the most important thing is I think in terms of decentralization what we already achieve, I can say is we like proactively sharing our contract repo. So all the contracts are quite easily accessible and in a nice format. We update regularly, weekly or daily. which was one thing missing in the ecosystem. And the other thing is of course the whole Sourcify is open source so you can easily run your Sourcify instance. But for us like the ultimate decentralization would be you don't need to like go to GitHub, download Sourcify and everything, but Sourcify can get, can easily be integrated or Sourcify's verification model can be integrated to block explorers or other applications that you just pull the files, compile on your computer, on your browser and then do the verification on your own. But yeah, in short it's still down the line.

M. Conor Svensson: And just, I mean with those sorts of things. And I appreciate we might be speculating too much but is it something that you'd envisage? There needs to be some modifications in the solidity side of things in order to cater for that. Or is it, or maybe services such as IPFS need to just be able to scale more easily or make it more readily available to obtain this contract metadata. Because I know that with the original vision I guess, of this with these metadata files it was about having these hashes of where the files live on a decentralized file storage network. But obviously given that the source file services exists, is I think testament to the fact that the original kind of plan is not, was not quite adequate, be that because of the technology not being mature enough or other more nuanced reasons, but is that kind of what that approach could look like down the line if we have that sort of decentralized file storage networks that can cope with the sorts of volumes you'd need for a service like this that can be verifying contracts every, you know, far more frequently, as you say.

Kaan: Yeah, for the IPFS thing I think there are a couple things I can comment on. One, this is so the IPFS support requires multiple layers. The biggest thing is someone needs to pin those and right now we do that. So we host the IPFS files, but then the contracts need to be perfectly verified. Which means I don't want to give too much detail but the content of the source code should not be touched at all. And this has been a problem to have consistent verification and have perfect verification. And that's why it's not always available on IPFS. Running an IPFS node is also not fun. So we are, we are still struggling on this and we are trying to outsource as much as we can on this front. And lastly, the consistency of the hashes has been a problem. That's why not many people like the IPFS hashes in the ecosystem. But also one thing to note is even if you get the sources from IPFS, using the IPFS hash, if you don't know, the solidity compiler appends an IPFS hash at the end of the bytecode and that is kind of the compilation fingerprint which can also contain the source files. Even if you get that, you still need to verify. So you still need to pull the files from that hash and compile and do everything. So at the end of the day I kind of realized it doesn't matter if you get the data from a centralized or decentralized source. The only thing would be that, okay, if source files used to exist, maybe exist, these files will be available on IPFS. That would be the positive, but otherwise it doesn't matter where you get the file from a centralized or decentralized data source. The important part is that this is being published and some people have these copies of the whole database. And Yeah. And to speculate a bit on this point. I have actually no idea how this can be done or if at all. But I think the ultimate endgame, if you call it an endgame, would be something like self verifying source code. So like if we, if there could be for example a ZK proof or something in the compiler. And the bytecode that can verify this source code is actually the bytecode of this contract. So this is just some speculation, but right now you need to go through the whole verification. But at the end of day this is verifiable computation, right? So maybe we might be able to somehow put a ZK proof somewhere and skip the whole computation.

M. Conor Svensson: Okay, this is interesting to hear, and certainly I think as you say, the challenges with some of the decentralized file storage infrastructure as well don't make life any easier there, so to speak. One of the things that was mentioned was the potential to support other smart contract languages. Would this be, I suppose the standard sort of, things like Viper, that might end up in scope? Or is there other sort of more esoteric languages that could come into the mix as well?

Marco: I think right now we are just focusing on Viper, probably because first we have to adapt our code to support more languages. But then for sure, once we have the viper integration, then I think it will be much easier to then integrate other languages.

M. Conor Svensson: And at this point in time, has work actually started on a Viper integration or is it more that it's going to be in the not too distant future?

Marco: We actually have some, we have active discussions about it and we are planning the next steps to, let's say we are in an experimenting phase, on viper. But then the problem is, for example, regarding Viper, we had many discussions with them, because we wanted to push them to integrate some sort of fingerprint for the compilation. Like for example the metadata, object, that we have in solidity. And so I think it's, we started this discussion with them, during the last Devcon in Bogota. I think I'm not wrong. No, maybe. No. Yeah, during Dev Connect in Istanbul. And so yeah, I mean, conversation, we also have an active discussion with them. Yeah. But now we're just in an experimental phase, I think.

Kaan: We can say it is scheduled for this quarter. Our old issues and all the progress and also the milestones and quarter plannings are also all public in our repo, in our GitHub project table. So everyone can check that out right now. We aim to have this ready at the end of this quarter.

M. Conor Svensson: Oh, awesome. One of the other things that you mentioned was the Verifier Alliance as well. So for the people who aren't familiar with it, is someone able just to give a quick overview of what it is and why it's sort of relevant for Sourcify.

Kaan: yeah, so the Verifier Alliance, as the name suggests, is an alliance from verifiers, such as Sourcify, Block Scout, Routescan, Tenderly and Dora. So these are other services that provide source code verification. The idea is to have basically a shared database schema and a shared database. And every one of these verifiers, whenever they have a contract verified, they push things, they push those contracts in this database, which we end up covering most of the contracts out there, hopefully. So we have been working together for a year, and we kind of completed our pilot phase and now we decided to do some more schema changes. We needed to change the database and add some more, let's say robustness to the data because the data ended up having a lot of mistakes. So right now we are transitioning to a new schema and hopefully we'll start publishing the data publicly as well on this site. So we already do this for Sourcify data but ideally we have Sourcify data plus all other verifier data in a nicely formatted exported dataset. And yeah we're still coordinating there.

Marco: Yeah and I think that another big advantage of being in this alliance is that there are a lot of niche cases while running a compiler that should be handled somehow and also like talking to people it's always like there is no proper documentation on how to tackle these bugs and niche cases while running the compiler. So something that I see that is fascinating about this alliance is that we are creating standards, so that we all manage these weird cases in the same way. So this standardization I think is another big advantage of being inside this alliance.

M. Conor Svensson: And does it mean the source of our service evolves a lot to almost be at the heart of that alliance given that there's a bread and butter so to speak of what Sourcify provides is this verification service and other sort of explorers, they either integrate with Sourcify or they have their own verification process. Does it put you so more sort of front and center there or is it more that Sourcify is just another contributor to this sort of infrastructure along with explorer providers?

Kaan: I think that's a fair thing to say, because our only focus is the verification. We obviously focus more on this and we have been more the pushing side let's say on these things. So we really want this to happen. I would say this, if this becomes a success this would be one of the biggest achievements of Sourcify being able to basically open source not only the code but also the whole data. So definitely we are  like one of the, I can say confidently, we are the ones pushing at this moment inside the alliance. but also the Sourcify's position as to how it's organized as a public good and non profit. It also I think gives us a degree of credibility, credible neutrality inside the organization so that we are more in a more positioned in a central place whereas some other members might be for profits and might have different incentives. Sourcify's position as a pure public good nonprofit I think makes it easier for us.

M. Conor Svensson: Yeah absolutely and it's just definitely interesting to see how it continues to evolve. So I think was it earlier this year that the alliance was announced initially?

Kaan: Yeah, it was conceived around last year, end of the summer, and earlier this year we announced it. And yeah, as I said, we are still working on it technically collecting data, but hopefully by the end of the year we will have a decent amount of data to be able to publish.

M. Conor Svensson: Awesome. And as you mentioned earlier on that, the roadmap is either currently available or about to be updated on the Sourcify GitHub. If people want to learn more about the project though, contribute to it or just, just get across it more. What resources would you recommend they check out first?

Kaan: Yeah, absolutely. Our docs, so if you want to get in, get more info about how Sourcify is different, how it works, how to run it, all of the things are available on our docs. if you really want to dive deeper, all our public repo, our issues, everything is open. And we try to use GitHub both as a place of documentation and also as a history of everything. I think this is quite important. We are documenting every problem and use cases or problems we have on Sourcify so people can look those up. Every now and then we have some design issues. Design issues meaning we ask for feedback from people. Right now, for example, we want feedback on our next API format. It is really, it is more than welcome. People have a look at that and give their feedback on our next API. otherwise they can always send us a message on discord or matrix if they want to get to know about something.

M. Conor Svensson: Well, we're at time now, so I'm just going to wrap up by saying Kaan, Marco and Manuel, thank you so much for joining us on the space today and look forward to continuing to follow and work with the team going forward.

Kaan: Thanks for having us, it was a great chat. Thanks.

Marco: Yeah, thanks for having us.