Unsure about how and why to use Source Generators in .NET? Stefan Schranz explains them in an easy-to-understand way. He compares them to T4 templates and explains that there’s a new and more powerful way to generate code. Learn about how to get started, the limitations, and their use cases in this episode.

Hello, and welcome to another episode of adventures in dot net. I'm Sean Klaver, your host. And with me today is our special guest, Stefan Schrons. Welcome, Stefan. Hey.

Thanks for having me. Hey. Thanks for coming. So, I think we're gonna talk about source code generators today. But be before we get to that, why don't you tell us a little bit about yourself, how you got into into development, and what kind of things you work on now?

As of now, I'm a software developer at TeamViewer. Basically, full stack developer working on dot net in the back end and also, the regular React TypeScript stack in the front end. Basically, I got into development kind of the the standout route, So through university, and I also had a tech background in my family. So my my dad used to be a system administrator at a local bank, and also my mom is a project manager at the at the same bank. So kind of was already a pretty fine path, And so I got into it, basically, the the the classic route, I would say.

So some people have Dockers in the family, and you're just a family of developers, I guess. Yeah. Basically. Basically. I can remember my dad trying to teach me c plus plus and pointers at 30.

Didn't quite work out, but I still managed my way somehow. Nice. So how'd you get into, dot net? Yeah. It was basically a bit of a wild ride until I got there.

So through university, I I went into university in a in a program which is common in Germany where you spent, 3 months in university and 3 months at a at a company just trying to get some practical experience. And, basically, I went through the whole cycle of c plus plus, hyphen, then pure front end developer, and some Java. But in the end, when I when I finally picked up my my my full position, then I got into a dotnet ASP NET team, and that's kind of where I spent my time so far, and I really, really like it. Alright. Cool.

So let's talk about, source code generators. Kinda have to go back to the beginning of them. Just kinda give a a an introduction of what they are and kinda what they're used for. It's because, you know, I've I've listened to other people talk about it, but they're still very confusing to me. Basically, source generators and and I just it's basically a new API which came in with dotnet5 and c sharp 9.

So you're probably familiar with a lot of approaches which were used ahead of this technology in order to generate source code. So there was, t 4, so text templating. And then maybe even even earlier, we we had to set up of having post build and pre build steps just, trying to generate some some source, which you can then use within your code. And, basically, with this new API, it's now possible to actually hook into the compilation cycle of your of your project and kind of also get everything set up without leaving any artifacts or without fluttering your your directory with new files. And, actually, you can also take advantage of a new thing with these source generators.

So you can basically hook into the compilation, and you actually get the entire pass syntax tree of your compilation, and you are able to to inspect everything in there and generate code accordingly. So is it is it kinda like the t four templates? It's something that you use during your development that's just gonna spit out some final code for you to put into your project, or is it something that actually can go at runtime? It's somewhere in between. So, t 4, as you already mentioned, it would actually spit out code, but you would usually get it in in actual CS files, depending on what you generate.

It's also not quite at runtime. It's in between. So it's actually happening in the compilation step. So when you would go ahead and start up your compilation cycle, you would basically start passing your your entire code. And then at this point, the source generator will kick in.

You will basically push your code at that point in time so that the past artifact, you would push it to the source generator. The source generator can then analyze it, generate new source code, and this new source code you generate will then go straight back into the compilation, and then the compilation will resume. And at the end, you will obviously get your your IL. So you're actually within the compilation cycle. So it's not during other events before or after compilation, but straight within.

So it's it's not gonna kick out a a CS file for you? It's something that that kicks out for the runtime to pick up and then go from there when you actually run it? It's possible to also emit it at the CS file, but, basically, it will just be there at runtime. The tooling is, I would say, a bit somewhere in between right now. So, if you if you run this generator once, you would usually also get IntelliSense support within Visual Studio for this because at that point, Visual Studio can also pick up the generated generated code, and you can also use it within your regular development cycle, get intelligence support and and whatnot.

But you will not have it on on disk by default. So it has some varying tooling support, and then it tends to feel like it's actually a source file within your project, but it's not. So what would kind of be a basic use for it? You know, with with, like, t four templates and things like that, you're gonna you you you build it, and you you run it. And then quite often, you have to go into the generated code and make some changes to be, you know, exactly what you want.

You know, if this is gonna run at compile time, then I guess you really have to get things exactly how you want it before you run that compilation. The very the varying of how you can use this. So, for me personally, when I experimented with it, I kind of set up a code base in which I I had, for example, a specific interface, which is inherited by some example classes. And then, for example, during compilation, I could use the source generator and create an a decorator for all of these classes implementing this interface, which would then maybe maybe log out something or to some additional things which might be common for all of these implementers. So that's maybe one use case.

So you can generate decorators and and other things like that. Basically, boring stuff you don't you don't want to write everywhere. Some people also use this to generate validation code on the fly based on some metadata. So, for example, you put some attributes on your classes or on your methods, and you can generate validation on on the fly. I think it's also really used a lot already within ASP NET, and some dependency injection frameworks already picked this up because being able to inspect the compilation tree on the on the fly, you can, for example, as a dependency injection framework, instead of relying on reflection at runtime, you can also take a look at the at the code during the compilation and basically see which classes need which dependency.

And instead of generating your dependencies at at run time, you can then already make use of it and have a look at this and during the compilation cycle. K. So, what's the first thing to know to get started, you know, writing your first source code generator? But there are basically some constraints right now. The biggest one is probably that, project containing this source generator needs to be done at standard 2.0.

So it's not yet on, net 5. That's basically a constraint of how the Roslyn compiler, which the source generators originally come from, is allowed to run. And then, basically, you need a setup containing of 1 project which hosts the generator, and then you will probably want to consume it from another project. So within the within the project containing the generator, you basically need to import 2 packages, which is the Microsoft code analysis c sharp and Microsoft code analysis analyzer. And then within the project, you want to reference it.

You would have the microsoft.net.compilers.toolset, package that will also bring in some IntelliSense magic. And then, basically, within the reference to this source generator, you would also have to put some specific text like output item type analyzer. And it's it's basically a bit of a setup, but, that's pretty much standout. And then once you do that, you can basically just create a class, put a generator attribute on top of it, and make it inherit the source generator interface, and that's basically it. It sounds a bit more complicated than it actually is in the end.

So it's actually pretty easy just, taking a look at the docs and then getting everything set up. And then, actually, it works quite well out of the box. So does it take writing a lot of code to get something that's gonna generate code for you? And, you know, sometimes, you know, you might be spending more time writing the generator than what you spent writing the code itself. It depends on the usage, I would say.

So, basically, a source generator always works in 2 steps. So there's an initialize method. And within this method, you can, for example, register your, code to respond to syntax note notification. So whenever the compiler picks up something new, you would have this event pushed to your, source generator, and you can maybe already analyze all the nodes whether they fulfill some constraint. But if you don't need this and you only decide, okay, I just want to generate code, that usually happens in the execute step of the generator.

Within there, basically, all you have to do is just as with regular code generation, create a string builder, and there you can write your c sharp code, and that's it already. So if you don't need any super special setup, you can just have your string builder, create a c sharp class, and you're good to go already. Okay. So is it kinda like using a string builder? You pass in some tokens and things like that you want to be replaced in that string as it compiles, and then that'll kick out the code for you?

Essentially, usually, very glorified string interpolation. And, that that's basically all there. So that's what you would mostly also end up with. Okay. Yeah.

That makes a lot more sense to me now that I think about it that way. It's just something that's gonna basically take a snippet or template of code that you have. And then, I guess, somewhere in your main code, you put, you know, like you said, you you'll use an attribute or something like that to to set it. And then in that attribute, you pass in the things that are gonna replace get replaced in that string builder? That's actually the beauty of it.

You can basically put anything anywhere. And since you get the the complete syntax tree of your program, you can analyze everything. So it doesn't even have to be an attribute. You can also inspect interfaces, check whether interfaces fulfill a specific name or whether they also implement other interfaces on top. Usually, attribute is is a good idea, I would say, because it's especially designed to represent some metadata.

So that's a good approach in my opinion, but you're definitely not constrained to attributes. You can you can mix and match as you want. So, basically you have it build a class, that class doesn't exist at the time that you're actually doing your development. So how does the other part of your code know that that's gonna be there when you compile? That's actually a bit difficult as of now.

So I think I already mentioned that the tooling support can feel a bit off here and there. Usually, it might take one cycle of building the source generator for, for example, Visual Studio to pick up the generated files at least. And then at least Visual Studio can give you a bit of a hand with the generated code and also give you some IntelliSense on top. But sometimes, for example, if you create a new generator and you never had it run before, then it can feel like you basically want to use that task, but IntelliSense just acts as if this would not exist, which it actually doesn't at this point. So, yes, tooling is a bit off, but I hope the, Visual Studio experience.

And I personally don't know how it how it behaves in rider, but I guess they would also have picked it up by now. So that part is usually entirely on the IDE, and depending on the support, it can feel good or mediocre, but usually, you can get by pretty pretty fine, I would say. So in that in that string builder, you know, as it's as as you're building that, telling it what the code you want to output, can it see into that code that you're building so that you can get IntelliSense and code validation and things like that? Not entirely sure, to be honest. I think there's also some tooling in in that direction.

I mean, can you do something like like a razor file, but be a be the the template of the code that you wanna use? Mhmm. I I think I I read up a bit. There are some extensions which also look into solving that problem. But I think out of the box, this is sadly not possible.

No? So you would kind of have to know where your classes are. Because in the end, when you when you create a a string that should then contain, like, a complete class, You would also have to take care of using namespaces and everything around that. So that's definitely something which feels a bit difficult sometimes because you have to remember all of these things, all of these boilerplate things you would also have to add on top. Yeah.

Because we if it did something like in a razor file or something like that where you could just do all your loops right within the little template to build it all up as the final string that you're you're trying to replace into your code, that would make it pretty easy to to work with, I think. You know, on that end, you can basically also just push a part of your class to the string builder, and then you can fall back to the regular c sharp code. And just, for example, insert a loop, which would then continuously append lines to the to the string builder. You're also not constrained to the string builder at all. You can also just use a very big interpolated string with with the at at the beginning to make it work across lines.

And you're pretty free to model that however you want. Do you find yourself creating these source code generators a lot for different projects? You know, I'm trying to I'm trying to think of a case where, you know, I really needed that that much other than, like, generating entities and things like that where, you know, Versus Code Visual Studio is already gonna help you out building those just like a t four template does. So when would you use a source code generator, and when would you use something like like a t four template? I think there's a very big distinctions with t four templates.

You are usually constrained to Visual Studio because I think they're much tied to that IDE. And with the source generators, you're actually running within Roslyn. So within the compiler, so you can also have it running on Linux or wherever you want. So that's a big difference, I would say. You are very much free to use it wherever you want.

When it comes to actual examples where I use this so, actually, I had a in this case for this a while ago. So at work, we we have an older project. It's c plus plus CLI. Maybe that rings a bell. It's it's a bit ancient.

And, basically, we have some some definitions in there, which we wanted to translate to completely manage, code models. And, basically, I also got started with this by using it as a as a post build step, you know, the old, wave generating code. But, I mean, in in my case, it was pretty simple. Colleagues started complaining at some point that I was just generating way too many files with that because I I I created a file per definitions, and there are quite a lot of them in that project. And, then I thought, okay.

This might be a good use case for a source generator. And, then I I did that, and it even worked flawlessly even if a in a dotnet framework code base. So not even if a dotnet 5. That's probably more of a convenience thing because there's, like, no technical reason to to have a source generator there. But I think I think I already talked about it a bit.

There are some use cases, especially within .net frameworks, which are very much focused on performance. So ASP NET is a prime use case for this. They already tried to move a lot of the code, which they would usually use reflection for at runtime and try to convert it to a source generator, especially if the information they would try to process, through reflection is already available at compilation time just to shave off that extra bit of performance during runtime. I think that's a pretty big use case where this is already used a lot. So I think people like me, being kinda being old school, the first thing I would go to is just make a make a copy of something that I want, and then I just copy and paste to make all the little changes and things like that rather than going to the hassle of writing a source code generator.

Do do you really have to get something that really has to be used over and over again? And can you can you write this generator and then share it between different projects? That's actually possible. So it's a bit difficult because I think I already mentioned that source code generators are kind of constrained to dotnetstandard to 0 right now. So that was definitely a bit of a surprise when I started getting into that technology because as with any good citizen, I already upgraded my, projects to dotnet5 at this point.

And I basically wanted to have the generator use some some specific names of some types. And then I noticed you can't reference it from a dotnetstandard project. You can't get a reference to dotnet5. So that was a bit of a shocker. So that's a constraint you have to keep in mind when you try to share these these source generators.

But, otherwise, it's it's definitely possible. And if you already use the new CS approach, format, so the one coming in with dotnetstandard, you would also get, the source generator output as a transitive dependency. So, for example, if you have a source generator project, which is referenced by another project, and then you would transitively add a reference to that project from somewhere else, you would get these, generated definitions everywhere. And that's already a nice way to share it. But besides that, you're also very much free to reference it from somewhere else.

That's just one thing to keep in mind because if you reference it from multiple positions in your code and you reference it in a way which triggers the source generator to actually generate them some source, then it's also pretty easy to end up with double generations. And you basically then have 2 conflicting types because the generator run twice, and, that's what you have in your project in the end. Oh, wow. Yeah. That's kind of a a a good gotcha there.

So alright. So I dotnetcenter2.0. So it's not in dotnet 6 either? Not yet, sadly. Actually, I also reached out to the to the Roslyn team on the c sharp Discord channel.

And as much as they would they would like to do it, as of now, it seems like they are constrained by, the support. So, dot net framework projects also support these source generators. So, basically, as of now, the best way they can do this is with dotnetstandard because you can consume dotnetstandard from dotnetframework projects and, dotnet5 or dotnet6 projects alike. I am certainly not aware whether there are any plans to also make it possible to reference, dotnet6 projects because that's currently a bit of a pain point, definitely, if if you want to just even if you just want to get a name of a type, you would want to check against this compilation tree, then it's a bit of a hassle. Yeah.

So, I mean, being at dot net Standard 2 point o only, you know, I think that could be kind of a discouragement for getting people to start using them. It's a new feature, but if they're not bringing it forward that that much, then people go, well, it's probably gonna get abandoned and and deprecated at some point in time. So why should I use it? That's, definitely a good point, but, basically, it's also not as bad as it might sound. So this whole generation step, I think you already talked about how the generation would create a syntax tree for you, and you can then analyze the syntax tree.

It's it's a bit difference. A bit of a difference than, with reflection because, for example, with reflection, you would also just go ahead and try to compare types or assemblies. And the syntax tree is basically one layer above, so you're not dealing with types at all. And you would usually just do it with with either strings, or you actually try to model your source generator in a way which can easily work with all of these syntax nodes. So it's not as bad as it sounds, to be honest.

And usually, the the benefits kind of outweigh the the drawbacks. So I'm also hoping, and I'm rooting for some improvements in that regard, especially when it comes to support for newer.net versions. But as of now, it is also not really big deal to me at least. Okay. Okay.

So, what are the limitations of what you can do and what you can't do? So talk about some things that you learned using them that that you would like to have known when you first got got started. There's a really big thing which I was definitely caught a bit off guard with. So as a person which did not yet use any ROSLAN APIs before, So Roslyn being the c sharp compiler, which also provides the source generators. They also power a lot of the suggestions within, for example, Visual Studio for some syntax improvements.

So just this thing of source analyzing is not new topic, coming to Roslyn. And they already seem to have a very established way of how you can inspect the syntax tree, which is very hard to get into as a as a beginner. And there are there's a very useful addition extension for Visual Studio, which is called syntax visualizer, and it basically allows you to step into your class. And on the side, you have a you have a very nice window which generates your your class as a as a syntax tree, and that way you can easily trace how you would also receive the syntax tree within your source generator. And that makes it way easier to to figure out which nodes do I need to check, which node will represent my interface, which node will represent my allocation in this case.

So this is something which is very useful to know ahead. Also, this feature is available on sharp lab IO. You can also change the result on sharplab. Io from, for example, c sharp to a syntax tree. And this is definitely something to keep in mind because that will be probably the topic which people will have the most trouble getting into.

It's definitely a different, domain than, reflection with with types. What other things should we, talk about about source code generators that that we haven't covered? I think we already got through most of what I think is important. So the I I think the the most important thing to keep in mind that is how how you can set it up, which constraint, it still has, which SDK version it can run on. And I think I already mentioned it right right away, the the syntax tree the syntax tree details, which are definitely a bit harder to get into.

I think besides that, it's something which is also something new. You have to wrap your head around this, coming to grips with how these source generators actually work, getting away from the thought model of having code generation ahead of the compilation or at runtime, but rather right within the compilation. That's also something to to keep in mind here. And besides that, honestly, I think that the technology is pretty nice, and, it's it's definitely something which kind of becomes what you make out of it. So there are not a lot of constraints as to what this thing can do for you.

That's probably more of a stretch of imagination then. K. So have they been making enhancements to source code generators over over time, or they kind of, just kinda have everything there they they think they need, and it's it's kinda be gonna be there? Are they gonna, you know, make have things changed in the future? There have been changes already.

I think when source generated started out, they were also similar to t 4 Visual Studio only thing, which already was improved on with dotnet 5. I'm not entirely sure what changed with dotnet 6. To me, it it still feels as if the technology is is some it's it's great. It's very useful, but it's also on at the at the same time, something which feels like was mostly developed for internal users. And I I think as of now, it's also the the main target the target audience for this technology.

So I'm not sure. I I hope there will be more changes to make it more friendly for the consumer, especially with tooling. Sometimes Visual Studio also, doesn't feel like picking up the generation at all, and that's also, very hard to deal with. So I hope there will be more changes in that direction. Sadly, I'm not entirely sure where the project stands right now, whether there are already some some enhancements in that area, but I'm very much rooting for it.

K. Where's the best place for people to go to to kinda learn and get started with the source code generators? The MSDN, documentation on source code generators is already pretty good, and it should it should definitely give a give an introduction to how how it works and how to set it up. And then afterwards, I would I would recommend just exploring the compilation, unit a bit. I think you already mentioned made it be on sharp lab IO or with, for example, the syntax visualizer extension for Visual Studio because that gives a lot of insights into what is possible, how can you get around your own passcode within the generator.

So these would be my go to steps in how to get into the technology. Okay. Cool. Any last things, that, you wanna let people know about before we move on to PIX? I think everything has been said already.

So I can definitely encourage people to just take a shot at it. I think just getting into new technology is always very nice, and this is one of those things things which feels like feels a necessary gap, which has just not been there yet. So I I think source generation has always been a concern of developers because, you know, developers, we are lazy people. We like to automate going stuff. And so I can definitely encourage people to check it out because it's it's a very nice and a very convenient and clean way to generate code without producing a lot of, garbage in your in your repository.

K. Well, I definitely have a better understanding of source code generators now that I've, you know, talked with you for half an hour or so. So great job at explaining source code generators to somebody like me that's kinda like been doing things one way forever, and it's like that's worked for me, so why should I do something different? So good job. So now we should I'll move on to picks, and I guess I'll go first with my pick for for this week.

And my pick is season 2 of a show on Netflix. It's season 2 of The Witcher, so I don't know if you, have Netflix or if you watched season 1 or are familiar with the things that where it came from. So season 2 just came out today. I haven't watched it, but based upon my watching of season 1, I expect it to be another good season. So I'll be watching that this weekend.

So everybody check out season 2 of The Witcher or rewatch season 1 if you, haven't watched it. I didn't watch it yet. I actually took the hands on way and played the Witcher instead on on Disney. Did you watch season 1? No.

Actually, I did not yet, sadly. You didn't wonder? I did I didn't yet find time to watch it. But I think now with holiday break coming up, maybe I can I can shave in a few a few hours of, watching The Witcher? Yeah.

Give it a shot. Give it a shot. Alright. So if there's is there something that you wanna, let our listeners know about that, you're interested in and want them to know about? Actually, I also have a very, very nice Netflix recommendation I could give.

So I'm I'm more of a documentation documentary watcher. So I'm not very much into into film movies or other other things like that. And I recently discovered that Explained on Netflix now has some new some new episodes on Explained, our our mind. And that's something I can very much recommend. So I'm I'm interested in, human psychology and just how our brain sometimes tends to play some tricks on us.

And, just getting some insights into that is is, very cool, and it's also presented in a very, very, very nice and bite sized way. Okay. Okay. Yeah. It looks good.

There's, 3 seasons of Explained on on Netflix. I've I think I've watched 1 or 2 of them and found them interesting, but I've, never had much time to go back and and check them out. So I will see all the different things they've they've came out with since the last time I watched it. Great. Alright, Stephane.

Is there some way that people can reach out and ask questions of you if they need to know more or get in touch with you? Sure. So usually always available on GitHub. So GitHub Sossenbinder. It's a bit hard because it's a German word, and there's no good pronunciation.

So spell it. So it's s o s s e n b I n d e r on GitHub. And besides that, I'm also on Twitter. So Twitter handle would be at.schranz. So that's at.d0tsch ranzed.

That would be probably my main channels. Alright. And that's that's z not zed for the, English speakers. So Yeah. Yeah.

Exactly. Yeah. I got it. Got it. Got it.

Alright. Great. Thanks, Stefan, for coming on the show. It's glad glad to have you here today. Thanks, though.

Listeners do wanna reach out and get in touch with the show, we'd love to hear from you. We would like to know what we're doing, wrong, what we could do better, and what kind of topics you would like to hear about. So you can get in touch with me. I am on Twitter. I am at dotnetsuperhero, and nobody else is here to go, so I'll do it.

Alright. Thanks, Stephan, again, and we'll catch everybody else on the next episode of adventures in dot net.