Thanks! Its true, many things are frustrating, like you say - bulk downloading. Yes DuckDB is amazing, and I use it in more than one of my projects. It has a lot of use cases for the data lake.
I agree, it would be more beneficial to query on patterns, but the main use case I saw for this app was non-tech folks wanting to see data in s3. I assume they wouldn't do fancy stuff like querying multiple files. But maybe I should add that functionality. Thanks for the suggestion!
A self-hosted, better browser interface to preview s3 files. This overcomes the limitation of the s3 console not allowing querying or viewing files beyond a certain size. It also gives you information about the column types. It could be a good use case to present this to non-IT teams in orgs who just want a preview of the data in the files or want to do some simple explorations. Frequently, they are denied access because of the possibility of doing something with confidential data via the s3 console.
For occasions like birthdays or Christmas where people want to give you gifts, I have always wanted to ask them to make donations to charities of my choosing instead. So I built an app to enable this: http://donateyourgift.com/ It is very simple but I didn't find anything quite singly-focused like it, so I built it just to scratch my own itch.
I was born and brought up in India, moved to the US at 27, but only now in my 50s chanced upon the amazing music that is qawwali. Till a couple of years ago I had no idea that such a treasure trove existed as part of Hindustani classical music (I was only familiar with the Indian side of Hindustani classical and the qawwalis in Indian films). I binge out on Coke Studio Pakistan songs and videos and find them mesmerizing. Fareed Ayaz and Mohammad Abu are treasures. Thank you for this, I’ll be sure to listen!
For occasions like birthdays or Christmas where people want to give you gifts, I have always wanted to ask them to make donations to charities of my choosing instead. So I built an app to enable this: https://apps.apple.com/ca/app/donate-your-gift/id6760786102
It is very simple but I didn't find anything quite singly-focused like it, so I built it just to scratch my own itch.
This looks amazing purely from an engineer's perspective that wants to level up on something they have worked on for a while with enough understanding to build working systems with it. Just confirming, this is more useful for developers that have worked with Erlang/Beam correct? Not so useful for a Beam newcomer?
Good for you then. I myself have not come across the (loud/louder) TDD exponents advocating for using TDD in system/integration testing, they mostly focus on unit tests. If you can point to some examples, it would be a learning experience for me. If not that's fine too, I am glad that there are voices out there like yours.
My experience is that TDD just ensures that the code is unit-testable. This can lead to more complex code when it need not be complex. I definitely write tests, but my default approach is to make the test simulate how a user would use the functionality. So mostly a higher level test like a system test. And you can do this only when you have a bigger picture of the program that you write (which need not be driven by unit tests, just need to elucidate what the program needs to do and break it into steps). I don't rule out unit tests, but my approach is to start with tests that resemble system tests, and if i specifically need a unit test for a hairy algorithm in a function, use a unit test to help.
Also the higher level you test at, the less probable that you have to change the tests when you change a piece of functionality.
I have pondered this question before and I have seen people recommending "Philosophy of software design" by John Ousterhout, but my qualms with Clean Code is not that it needs a substitute, its just that its a fairly simple set of concepts about which Bob makes a big deal. I did read some of his books, but I realized its only about 10% of what makes a competent software engineer. My suggestions to people starting out or even seasoned programmers are that get an idea of what he advocates (TDD, SOLID and all that) but then design of programs is just a small part.(And I also can debate the usefulness of both TDD and SOLID. Personal opinion coming: they are great for small or greenfield projects but almost always don't hold up in the real world).
Learn about other kinds of (much more effective) testing like System/Integration testing, Property-based testing. Spend a lot of time learning about databases and SQL. Maybe get into somewhat esoteric topics like constraint solvers and logic programming. You may not use these but it helps to know there's a wide world out there, and they do bend your brain enough to enable you to think differently.
Time is limited. It does matter what we spend it on.
I use it on my 2015 Macbook pro. Its amazing how quickly you can get set up, kudos to the authors. Its a dog in terms of response time for questions, but that's expected with my machine configuration.
Also they have Python (and less relevant to me) Javascript libraries. So I assume you dont have to go through LangChain anymore.
Thanks for the talk link, it completely kept me engrossed and was a pleasure to watch! Addendum question: what is the state of database drivers for Elixir? Does it have mature libraries for Postgres for eg? I imagine if it's targeting web development, it will need those.
You should look at the command `python -m venv`. Its built in, and is a breeze to create virtual environments. I guess it does not provide shortcuts for activation etc, but I am ok with that.
I thought one of the advantages of pyenv was that you switched to also creating virtual envs using pyenv and managed virtual environments with it too. Maybe not.
I used pyenv in my previous company. At first it was fine, but after a while with multiple versions of Python installed, things stopped working. The right virtual env was not activated, etc. (It could have also been because our software updates like OS and security upgrades were pushed by desktop support). I removed all of it, and just resorted to installing multiple versions of Python the old way (downloading from python.org) and then use `python<version> -m venv /my/virtual/env` to manage my virtual environments. Things are more stable, and I don't feel like its magic. I am not going back to pyenv.
Amidst all these great discussions, I would like to point out that this article really helped me get my head around a B tree and why its a great optimization on top of the Binary Search Tree. Thanks to the author!
http://cgopalan.github.io http://github.com/cgopalan
Email: [email protected]