This is really cool and promising product! So many times when I was getting a google docs spreadsheet from non-engineering folks I was wondering if only I can embed a small jupyter notebook with a few python cells and a nice looking pandas chart, using this data.. Congrats on the launch!
Sure, I agree, above certain usage threshold hosted Trino becomes totally justified. But then, some engineering time to maintain the cluster has to be factored in as well.. for anything ad-hoc in nature, I would start with Athena by default.
other arguments aside .. Athena costs $5 per 1TB scanned and also supports predicates pushdown to S3 Select. I wouldn't call this expensive, at least in comparison to self hosted Presto.
For this particular example direct initialization i.e. std::string product("not worked"); would be preferred, as you end up using one call to constructor instead of two: default constructor followed by the move assignment.
Good progress overall, especially on the Rust side, I wonder if C++ and Rust would at some point follow the same roadmap when it comes higher-level compute features or rather deviate and develop at their own pace.
Special kudos to the Rust team for Parquet predicates pushdown feature.
That's correct, but SQLite would require to serialize/deserialize the data sent to Python func (from C to Python and back), while Arrow allows to get a "view" of the same data without making a copy. Which is probably not an issue in OLTP workloads, but may become more visible in OLAP.
Following similar observations I was wondering if one can actually execute SQL queries inside of a Python process with the access to native Python functions and Numpy as UDFs. Thanks to Apache Arrow one can essentially combine DataFrame API with SQL within data analysis workflows, without the need to copy the data and write operators in a mix of C++ and Python, all within the confines of the same Python process.
So I implemented Vinum, which allows to execute queries which may invoke Numpy or Python functions as UDFs available to the interpreter.
For example: "SELECT value, np.log(value) FROM t WHERE ..".
Finally, DuckDB makes a great progress integrating pandas dataframes into the API, with UDFs support coming soon. I would certainly recommend giving it a shot for OLAP workflows.
That's one of the core use cases for Vinum - to provide SQL query engine, complementing Pandas in data analysis (thanks to Arrow), yet retain an ability to use native python and numpy funcs as UDFs. Also, it doesn't require input dataset to fit into memory.
HERE | Senior Software Engineer | 60-75k EUR | Berlin, Germany | Full-Time | ONSITE | VISA
My team, HERE Datalens is looking for great backend engineers to help us crunch and visualize big geospatial datasets. More about our product here: https://goo.gl/gcj8h3. About HERE in general: http://here.com/
We are looking for strong backend engineers, ideally with experience in crunching big datasets. Exposure to GIS/Maps is a big plus. We mostly use python in our stack, but depending on the task may switch to Java or C++, so we don’t expect python expertise, but you should be proficient in at least one of modern languages/ecosystems. Good grasp of algorithms and ability to reason about backend architectures is required.
Our team is relatively small and we try to operate as a startup. This is a full-time position based in Berlin and we do provide relocation package for those willing to move here.
This is a full position description on StackOverflow https://goo.gl/N1pDSp. Please feel free to apply there or send me your cv directly - dmitry.koval at company name com
It always amazes me how people keep recommending CLRS as an interview preparation material. If you haven't studied Algorithms before it will take months to go through material in this book, leave alone making exercises. It is a serious textbook with strong mathematical rigor. If you have studied algos, granted, you would know what book to pick as a refresher and chances are - CLRS would be on your table.
I think Sedgewick's book would be a more practical choice as a first book to learn some data structures/algorithms.
If you happen to be asked such questions it might be a good indication that the type they are looking for is code monkey. Or they are just not good enough to assess engineering skills. In both cases it's a red flag.
It seems you like you still hire based on puzzle as a first filter (based on on your careers page). Just wondering, what is the ratio of people who apply with only a resume vs those who solve the puzzle (I understand you probably can't reveal). And, more importantly, are those on the puzzles side show to be more successful on the interview?