Chatting with Sebastian Witowski - Part II: Python Community and Developer Tools

Sebastian's work in the Python community: talks, articles, courses. And a favorite topic: developer tools and productivity.

Date

Nov 04, 2022

Sebastian giving a lightning talk about developer productivity at PyCon Taiwan Jim_7596(CC BY 2.0) by PyCon Taiwan Official Album

This is the second part of our interview with Sebastian Witowski. In case you missed it, here's a link to Part 1. Today, we talked with Sebastian about contributing to the developer community, focusing on his talks and writing. We also discussed one of his frequent topics, developer tools.

At the end of the first interview, Sebastian mentioned that he picked up Python during his work at CERN. Let's continue the conversation there.

Reka: Can you tell a bit more about your time at CERN?

Sebastian: One of the nicest things is that you have people from all around the world. Even though I was not directly involved with any of the experiments, it was nice to be part of what's happening there.

I worked in the IT department on an open source software, Invenio, to create digital libraries. So if you have a stack of books or some CDs or whatever and you want to organize them, build a digital library or maybe rent them to people, you can use Invenio to set up your own digital repository. You can also categorize the content, add some metadata, etc.

It was really nice to see that what you're doing is useful to people. We were giving talks about Invenio at conferences. We had users coming to CERN from various institutions around the world to learn how to use it. And we saw more and more libraries and universities use it.

There are also several projects based on Invenio. Obviously, there's CERN Document Server for all the papers produced at CERN. And not only physics papers but also some fascinating archive photos. One of them showed a guy on a bike transferring huge floppy disks between CERN buildings 60 years ago. That's how networking worked back then. :-) There's another project called Zenodo, funded by the European Union. It's for researchers who want to share their datasets but can't afford to pay for Amazon S3 or a similar service. This way, they could make petabytes of data available for the public.

CERN Large Hadron Collider
Chris Mitchell, CC BY-SA 4.0 <https://creativecommons.org/licenses/by-sa/4.0>, via Wikimedia Commons

R: How is such an open source project different from the usual enterprise projects?

S: It was an interesting challenge. Most of the contributors were working at CERN or one of the universities. But then we also had external contributors. I guess it's similar to Microsoft working on VS Code or some other open source projects that they support.

You have a dedicated team of people who triage the issues and draw the roadmap. And you have external contributors interested, for example, in implementing a specific feature. Some of them will stay for longer and some will just contribute one thing.

Overall, it was working very smoothly, I would say.

R: What about the collaboration part? How is that different in a company and in open source?

S: If you have a team of developers, you can have a meeting, you can have a presentation for them, and then you can discuss face to face. So it's much easier to collaborate and set common practices. If you're working on an open source, you're not gonna call your contributors. And there is a slim chance that they would watch a video if you sent them.

So here you need to put more effort into writing a nice README and setting up the CI pipelines. To make sure that everything is checked and it doesn't annoy the contributors but also brings enough benefits for them.

So I think the main difference is that one is more asynchronous and the other is more synchronous communication.

R: Are you still in touch with Invenio?

S: They are going strong. They have more and more collaboration with different universities. But I don't participate in the project actively.

I usually don't contribute to open source. It doesn't really work for me to code at night after the whole day of work because I was, like, getting huge headaches and migraines.

And I said, okay, I just can't do this. Now, I try to give at least one new presentation at conferences per year. That's my way of giving back to the open source community.

Sebastian giving a talk at EuroPython 2022
Photo by Larissa Oliveira on EuroPython 2022 official photos

R: That's a cool way of giving back, indeed. Can you tell about your process and how you create content?

S: I start by taking some notes. If I have some ideas for a blog post or a talk, I write it down. And then if I get enough notes on it and they can clarify into something I can talk about, I make a proposal. But for example, I have no idea what I could talk about next year.

When I decide I'm gonna do a talk, I start preparing it way in advance. I usually have a proposal and outline prepared even before I get confirmation from the conference. Because when I know I'm gonna talk about a specific topic, this topic stays in the back of my head.

And then whenever I see an interesting article that would apply to it or I have some ideas, I just keep adding them to my notes. And suddenly I have, like, fifty links or resources.

Then I can very easily build my talk from that. There is no way I would be able to build a whole interesting talk in two weeks without having any materials prepared.

R: You've given talks and created a course about developer tools. How did you come to this topic?

S: I'm always interested in tools. If there is a way that I can improve my productivity, I'm very keen on testing them. But I'm not some productivity junkie that jumps on shiny new toys. I'm quite conservative and I have to hear about a given tool a couple of times or it has to solve a big problem that I have at hand.

At some point, I realized there's a workflow I like using. pyenv, pip-tools and so on.

And then I realized that there were actually no talks about that. There's a lot of material about Python basics, about advanced topics or focusing on a specific library. But there were no talks about how you write a Python project. Like, how do you configure your editor? What could be some useful tools to have or at least know about?

Perhaps you use pytest, but you don't know half of its features like fixtures. Or similar with Sphinx. So many low-hanging fruits that can make your documentation much better, like testing the docstrings. Or the plugin that enables you to click the function name and go to the API documentation. Those are, like, one-liners in the configuration of Sphinx. But if you don't know this exists, you're probably not gonna search for it. Once you know that it exists, it makes sense for you to use it everywhere.

So I came up with, like, a list of tools that I would use, and I think it would be useful to at least know about, and I made a course about that.

R: What feedback did you get for this course?

S: I think it was very well received. The initial, free version was a 2-hour workshop for PyCon US in 2020. And I think this is still the second most watched video from that conference. So people really like it. I also made a website with the material, and people keep saying that it's nice. I still see people going to this website and sometimes it reappears on HackerNews.

Then I mentioned to Michael Kennedy from Talk Python To Me that I was thinking about making a longer version of this course. We started talking and I ended up producing the whole course with them. And a bunch of people said it was very useful.

I got some people not happy with the choice of tools. Recently, someone posted an issue on the repository saying this course should use Conda because of a list of reasons. Of course, that's the problem when you tell people about tools. Some will be like "No, this tool is wrong. I'm using this other tool, and it's better." It's a difficult topic because people are opinionated. And again, often, the more senior someone is, the more opinionated they are. Luckily this is not a rule.

I tried to structure the course so that the tools are most universal. And replaceable. I wanted to avoid showing one tool to do everything. This way, you can change a part of the workflow and still find the rest of the course useful.

Modern Python Projects Course on talkpython.fm

R: This course was published one and a half years ago. Is there any tool that you would change now?

S: Not really. pipx, venv, pyenv are still there. cookiecutter is still the most common tool to create project templates. Sphinx for documentation, pytest for testing. CI pipelines with GitLab and GitHub, Docker containers.

If I were to record this course today, I think I would use the same tools. Some commands might not work, but I think the content of this course is still relevant a year and a half later.

R: You mentioned that many people are very opinionated about their tools. What about those folks who aren't that much into tools? What are the "basics" that even they should check out?

S: I guess it depends on what pain points this person has. Let's say I know that this person is working with multiple Python versions. And they have all those different Python versions just installed on their computer, but sometimes one of them breaks. Then I could recommend pyenv or a similar solution to that. Because it's hard to sell someone on a specific tool unless this solves a problem that person has.

Or, for example, maybe someone is installing pip packages globally. And if you do this way too many times, then suddenly you have a complete mess. Then I would say look, you can use pipx. This way, the packages will be available globally but live in separate environments.

Or if someone's working on a project without pinned dependencies, I'll mention pip-tools. With pip-tools, you can pin the dependencies. And the next time you send your project to someone else, they will have the same versions of dependencies.

R: You've also written a lot about CLI tools. What are your major recommendations there?

S: Let's start with the shell. If you don't want something too crazy, you can go with Bash or Z shell. I use fish shell, which has a lot of nice features built-in without any plugins. But it's quite different from bash and zsh, so sometimes, if you want to run some Bash scripts, it won't work out of the box.

Starship screenshot
Sebastian Witowski: My Favorite CLI Tools https://switowski.com/blog/favorite-cli-tools/#starship

Something that works really nice for me is a tool called Starship. You install it inside your terminal, and it intelligently changes the prompt. So if you are in a git repository, it will show you the current branch. Or whether you have different changes in your local repository than in the remote repository. If you switch to a folder where you have a Python project, it will show you which version of Python is being used and so on.

A nice thing about most of the tools that I'm using is that they work out of the box. Starship requires absolutely no configuration. And it gives you only relevant information for your current setup. So it won't, let's say, display the Node version if you're not even in a Node project.

For jumping around the file system, I use zoxide or formerly z. It works super smart because all you have to do is to use your computer normall. And then zoxide will learn what the most common folders that you go to are. You type "z" and a few letters from the name of the folder, press enter, and you'll immediately go to that folder. If you type "z down", it will go to your downloads folder and so on.

These are all small tools. They do one thing and do it very well. I really can't imagine life without them. I don't remember when was the last time I typed the whole path of something from my home folder. I just jump around, and it's much easier.

There are also some tools that replace the built-in Linux tools. They are written in Rust and are usually faster than the built-in version. They look nicer, have colors and very good defaults. For example, ripgrep will ignore the files that you have in your gitignore, your node modules and so on.

For some of them, I just alias the original Linux command with this new version. So when I type grep, I would use the ripgrep instead of the standard grep.

R: In a recent blog post, you mentioned that for static sites, you usually use 11ty now. How did you choose that?

S: I often create tiny pages, like a landing page for my course. Or a blog with pictures of my family. I realized that I have many pictures, but only some of them are worth going back to in the future. So I devoted some time to select some pictures and descriptions so I can look back 20 years from now and see the best moments of my life.

I used to pick Jekyll for such sites, but now I feel it isn't as actively maintained. It also took more and more time to generate the page. Especially for this family blog, because resizing the pictures takes time.

Then I found 11ty, and I really like that it's minimalistic and composable. If I want to have images or an RSS feed, I just add the respective plug-in. It's also super fast and fun to use.

R: How do you learn about new tools? You've mentioned that if you don't know that something exists, you won't search for it. What are your strategies to learn what is out there?

S: That's a very good question. HackerNews is an excellent resource, even though I don't browse it that often. But usually, when there is a nice tool, it will appear over and over.

I also really like blog posts with productivity tips or a list of tools. Like, a list of the best Rust CLI tools that you should be using. You can just read this list, pick a few tools and test them out. And maybe some will really stick with you.

Another source of information is the Python Bytes podcast by Michael Kennedy and Brian Okken. The tools they mention often resonate with me. Just the other day, they mentioned difftastic, and today it's already installed on my computer. So, it's completely random. Some blog posts, some HackerNews stuff here and there when I have some minutes. And I usually listen to podcasts when I go to the gym. I think tech podcasts are the most common way for me to stay updated with the technology.

R: Which tech podcasts do you listen to?

S: Mostly Talk Python To Me and Python Bytes. I really like Python Bytes, because they're short but have like five different topics. So it's a really great way to stay on top of what's happening in the Python ecosystem.

R: Do you have some not Python-specific podcast recommendations?

S: I would mention Rework, where David Heinemeier Hansson and Jason Fried talk about topics of their book.

A completely nontechnical favorite of mine is the Huberman Lab. This is hosted by Andrew Huberman, a neurobiology professor at Stanford University. He talks about science and gives practical advice, I think, in a trustworthy way. He always tries to mention only peer-reviewed sources from established journals.

For example, I got hooked when he was talking about eyesight. Because I'm staring at my screen, and my eyesight gets bad. And he had some nice advice on how you can improve your eyesight.

Or he had a whole episode on how you can improve your sleep. Like taking magnesium in the evening. Or getting some sunlight in the morning in your eyes because it regulates your circadian rhythm.

EuroPython 2022
Photo by Larissa Oliveira on EuroPython 2022 official photos

R: How do you pick which events you attend?

S: My favorite one is EuroPython. That was the first conference I went to in 2016, and I've already been there six times. At some point, when you start going to a specific conference, you keep meeting the same people as in previous years. So now I go there mostly for the people I know. You have those colleagues that you meet once per year, and then you just resume the conversation you had a year ago. It's always fun.

Usually, I go to EuroPython and to one or two other conferences in Europe. I try to go to new conferences to see how they are organized. I went to PyCon Italia this year, and it was great. Lovely people, amazing place. Also, the conference was happening in the hotel, so everything was at one place, which was very convenient.

This year, I will also be talking at the PyCon Poland. When I moved back to Poland, I was travelling when the conference happened, then we had COVID, and this is the first time ever I'm attending the PyCon happening in my home country.

During the pandemic, it was very easy to talk at a lot of conferences because everything was happening remotely. On the one hand, that was nice because I spoke at the PyCon Australia, Hong Kong, India, Sweden, and so on. On the other hand, I'm not a huge fan of online conferences because the networking there doesn't work at all. At some point, it felt like just a chore. I work in the morning, then I log in to the conferencing page and I give my talk, I answer the questions, and then I go back to work.

R: Do you plan to do more courses?

S: Making a course was definitely an interesting experience, and it was a nice change of gears from my usual daily work. But it also took way more time than I imagined because I'm usually very thorough, and I wanted it to be perfect.

So I spent a lot of time preparing the material, reshooting the videos and so on. Initially, I thought, okay, I'm gonna do this course in, like, two months maximum. Yeah, right. It took me over half a year. But I was actually between the projects, so I was kind of relaxed about that.

I learned quite a lot of things that I'll do differently next time. I don't have any specific ideas, but I definitely want to make some new courses in the future.

R: By the time you made this course, you had given several talks and written lots of blog posts. Do they also sometimes take longer than expected?

S: It depends. I got better with time, but writing blog posts still takes a lot of time for me.

When I prepare a talk, I gather all the materials. I write an outline, and then I can sit down and just write everything I would like to say. So with all this preparation in place, I could sit down and write the whole talk in one day.

Initially, when I was doing that, I vastly overshot with the amount of content I wrote compared to the amount of time I had in my time slots. So if I prepared a 30 minutes talk, the first version of it was over one hour.

Now, I have a life hack for this. I measured that when I speak at a conference, my pace is 150 words per minute. So I know if I have a 30 minutes slot, that's 4500 words, and that's the ballpark where I'm aiming with my talk.

Occasionally, it still happens that I have too much material. This year, I've given the talk Writing Faster Python 3. The first version was over one hour. And the first time I presented it at PyCon Italia, I had a 30-minute slot.

So I had to remove a lot of content. And in the end, I was unhappy with the results. So, now I propose it for a 45 minutes long slot, and that's much better.

Writing Faster Python 3

R: What are these things you'll do differently for the next course?

S: There are several things which I changed while I was creating this course.

I started recording with my face, and then I wanted it to be perfect. The first ten minutes of the video took me half a day because I was reshooting and reshooting. Then I realized that it's kind of stupid to have your face in the video because no one cares.

I set up a separate account on my computer for the recording. Removed the wallpaper, most of the toolbar icons and so on.

At some point, I also created a checklist of all the things I have to set for the recording. The font of the terminal, the gain in my microphone and so on.

One error I did was that at the beginning, when I was making a mistake in the video, I was just stopping and restarting. Which is super stupid because you can cut it out in production. Just keep on talking, and then during the editing process, you can remove it.

I also learned several small tricks. If you want to show something important on the screen, you can actually zoom instead of moving your mouse over there.

Don't sweat about the equipment that you have. I had this seventy bucks microphone, and it turned out to be fine. You can also use a noise-cancelling service. I used a paid one, but I have a mechanical keyboard, and sometimes the noise is loud.

There are many small things that you learn as you go. But when you compare your first video to your last one, you will notice the difference.

R: How does your workflow for writing look like?

S: I usually take a lot of notes in Obsidian. Not just for blog posts but everything. I have a shortcut on my keyboard to open Obsidian. It's like "Quick Notes" on Mac, but in Markdown.

I also have the Obsidian app on my phone. So if I have an idea, I can add it to the respective note there.

Then when it comes to editing a blog post and preparing it for publishing, I use VSCode.

I'm actually working on a blog post about my workflow in Obsidian.

R: That's awesome; I'll definitely check it out. Thanks a lot for the long conversation and the awesome tips.

Sebastian Online

Resources

Some Articles by Sebastian

Some Talks by Sebastian

Podcasts

Conferences

Other

Tools

Python-specific:

General CLI tools:

Static site generators: