TLDR: SQL feels most natural for joining and aggregating data sets; Python is favorable for filtering and transforming data due to its higher flexibility and extendability
DataCater (https://datacater.io) | Remote (Germany or CET +- 2h) | Full Time | Cloud infrastructure engineers, Frontend engineers, Software engineers
At DataCater, we help businesses to efficiently connect their growing amount of data silos. To this end, we build the self-service platform for streaming data pipelines, based on the open-source technology Apache Kafka. DataCater enables companies to benefit from the power of event streaming, without handling the complexity. Our mission is to provide essential tooling, so that data teams can unlock the full value of their data, faster. DataCater was founded in 2020 and is serving customers from industries, such as finance, e-commerce, media, and software.
Sounds interesting? Please send an e-mail to our CTO Hakan ([email protected]), tell a few words about yourself, describe what excites you the most about the role, and send along your CV.
I suggest to try thinking in terms of events instead of streams.
You can extract data change events (e.g., INSERTs) from a data source, transform them with a streaming application (e.g., built with Kafka Streams), and load the transformed events into a data sink.
DataCater (https://datacater.io) | Remote (Germany or CET +- 2h) | Full Time | Cloud infrastructure engineers, Frontend engineers, Software engineers
At DataCater, we help businesses to efficiently connect their growing amount of data silos. To this end, we build the self-service platform for streaming data pipelines, based on the open-source technology Apache Kafka. DataCater enables companies to benefit from the power of event streaming, without handling the complexity. Our mission is to provide essential tooling, so that data teams can unlock the full value of their data, faster. DataCater was founded in 2020 and is serving customers from industries, such as finance, e-commerce, media, and software.
Sounds interesting? Please send an e-mail to our CTO Hakan ([email protected]), tell a few words about yourself, describe what excites you the most about the role, and send along your CV.
DataCater (https://datacater.io) | Remote (Germany or CET +- 2h) | Full Time | Cloud infrastructure engineers, Frontend engineers, Software engineers
At DataCater, we help businesses to efficiently connect their growing amount of data silos. To this end, we build the self-service platform for streaming data pipelines, based on the open-source technology Apache Kafka. DataCater enables companies to benefit from the power of event streaming, without handling the complexity. Our mission is to provide essential tooling, so that data teams can unlock the full value of their data, faster. DataCater was founded in 2020 and is serving customers from industries, such as finance, e-commerce, media, and software.
Sounds interesting? Please send an e-mail to our CTO Hakan ([email protected]), tell a few words about yourself, describe what excites you the most about the role, and send along your CV.
DataCater (https://datacater.io) | Remote (Germany or CET +- 2h) | Full Time | Cloud infrastructure engineers, Frontend engineers, Software engineers
At DataCater, we help businesses to efficiently connect their growing amount of data silos. To this end, we build the self-service platform for streaming data pipelines, based on the open-source technology Apache Kafka. DataCater enables companies to benefit from the power of event streaming, without handling the complexity. Our mission is to provide essential tooling, so that data teams can unlock the full value of their data, faster. DataCater was founded in 2020 and is serving customers from industries, such as finance, e-commerce, media, and software.
Sounds interesting? Please send an e-mail to our CTO Hakan ([email protected]), tell a few words about yourself, describe what excites you the most about the role, and send along your CV.
Thank you! :) I updated the article with a note on privileged containers.
Evaluating user code inside privileged containers is indeed a security nightmare. Fortunately, --privileged is not enabled by default, which is why I think that containers are quite secure by default.
TLDR: SQL feels most natural for joining and aggregating data sets; Python is favorable for filtering and transforming data due to its higher flexibility and extendability