![]() So in case you're in a company, you like to make data driven decisions, because data driven decisions, that's what makes a company, gives a company competitive advantage. But why do we need such this existing data pipelines? Why is that actually important? A little bit of theory. And I think in that case, we're a bit in for a bit of new things. So it's about having an appealing data pipeline. But not when I'm around with the data pipeline, I'm not sure that's going to work. If you want to get to the first side, be my guest. Your own download dating, that's fine, but we're not there yet. I assume it's about the second definition that we're talking about appealing, not eroticism, nothing like that. We do not care when it resides and what you look like, but we do like a sexy data pipeline for the ones that you are more English than I am. In this case, I hope it's gender neutral, because we go both ways, right? Or always, whatever you want. So we'd like to go to Tinder for new data pipelines. Modern times especially, we just ran out of the pandemic. And typically you do that on a dating app nowadays. If you feel like having a friendly, it's okay. If it comes very much like a frenzy, which could happen, I will create a break inside my talk. Don't hesitate to go to the back and grab one of those pizzas. I understood there might be pizzas arriving. But like I said, everything is available to you. And I'll sprinkle a little bit with astronomer, as I explained. But today I will be talking about sexy data pipelines. We are very welcoming community in that way, and obviously your code will be scrutinized, but we try to look at it within a positive way, and that you try to scratch it if you try to solve the socket. That's also the inviting, the invitation to you, by the way, for the future, if you'd like to get involved. So that was kind of compared to where we were at. And Maxim was the guy we invented Airflow and now runs Superset, kind of merged it right away.Īnd I was like, what did you actually look, what I was writing was because what's my first Python code ever? And then we put it into production at Airbnb, and they were running at that time about 60,000 tasks per month. So I did kind of a pull request of about 50,000 lines changed. And I started looking at that thing from Airbnb. ![]() You need to look at that thing from Airbnb. And I said, there's something called the Wii. I became involved with Airflow in 2015 because of one of my data scientists was saying, yeah, we need something to schedule our workloads with. But I also know it from the engineering side. So I know what the uses, the output of data pipelines is and putting those into a product. I've been the head of an analytics department for about 150 people where we created data analytics platform products in wholesale banking and also retail banking. Before that I used to work for IG Bank for 14 years. But before that I only worked there since one and a half years. I do work for a company called Astronomer that you might have heard of through different Airflow meetups. I'm not going to give an extremely technical talk if that is what you expected, but everything what I'll be showing is going to be available to you. Obviously a little bit of a teacher teaches her, but I hope you'll have fun, enjoy doing it and enjoying with me a little bit of this talk. I like to be close to and today, yes, I'm going to talk about sexy data pipelines. I need to make it a bit nice here as well. Different sizes too, so don't worry about it. Or are you going to be shy? Go free t shirts. Do you want to sit down next to Castle? It's fine. I hope you have to be fast though, because there's little space left. You're going to be in front, right? You promise? Here you go. I know you're very afraid of being in front, but I can offer you some free t shirts. Before I start, I see some empty couches in front. Now he's the VP of the project and he's a main contributor as well. He was the second person to contribute to the project since then. He's been contributing Chapatch airflow since 2015. Well, our speaker today flew from the Netherlands, boke So. And thank you very much for making this space available even and. He has a company at working and his office is here in Accelerator. If you have any venue and you would like to host, we would very much appreciate. There's a chance the next meetup will be made. Since the Pandemic we had to think some online meetings and then we are trying to get back on track. So before the Pandemic we would have regular meetups. He's one of Airflow PMC members, one of the top committees of the project, together with Vogue who was the second contributor to Chapat Shareflow and Ash as well, because another PMC member. It was asserted by Paxo, who just by me left.
0 Comments
Leave a Reply. |
AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |