Commenting just to go against the other two answers. I think it's fine to not remember things, no matter the apparent simplicity.
Quite surprised by others finding this as a... Surprise? I get there is people who never experience this, but they also not know anyone personally to whom this would happen?
What does data frames mean in this context? I'm used to them in spark or pandas but does this relate to something in how duckDB operates or is it something else?
Without any context of culture or country, just trying to be helpful: in my limited (<20 total interviews) experience, I would think about budget issues.
Meaning, what you ask for (or how expensive you are perceived, if you have that strong resumee) for the industry you apply, may be too different and leading to limited access.
Sometimes I feel junior people have it easier (I felt like I did, personally) since the expense in salary is pretty limited compared to either other roles or more senior people
It's been mentioned but I want to add that the original idea of the post (mid size VPS hosting apache spark) might be missing that spark is ideal for distributed and resilient work (if a node fails the framework is able to avoid losing that work).
If you don't need this features, specially the distributed one, going tall (single instance with high capacity, replicate when necessary) or going simpler (multiple servers but without spark coordinating the work) could be good options depending on your/the team's knowledge
Recently I have been thinking about this, because I feel I have managed to become way more organized than I ever thought it was possible.
What is working for me right now is noting everything in a calendar so I cannot forget it or as TODO in a somewhat heavy personalized Obsidian configuration.
A few years ago (5-6 aprox) I started copying my older co-workers habits to see myself improve. Physical notebooks were soon discarded because I never remember where I wrote down things.
I used a TODO plugin in sublime which worked for several months, until I felt I needed screenshots so I moved to OneNote. After a while I became frustrated with not being able to customize it enough, so I started trying out different things. I saw a coworker using Obsidian, watched a couple long YouTube videos to learn how to customize, and I'm never going back.
My team this week told me they are impressed with how much info I write down and it was a very proud moment for me!
PyIceberg is nice but we had to drop it because it's behind Java API and it's unclear when it will match up, so depending on which features are needed I'd look it up
As someone doing this transition, I would love some references that help me... Train myself I guess? Other than by doing and analyzing myself, which is my current situation
I have realized I can give so many tips and reference so many great content online to learn math, programming, engineering... But find myself missing anything about managing
Since I feel the same, and my habits are really similar to yours (I'm forcing myself to read more articles compared to when I used reddit), I want to comment something I felt lacked in the article analysis.
For me, what's missing is intent. Most of the time I feel I could be doing better, I stop and think: wait, this is my resting time. Or, wait, this is my hobby. I feel even though I fall in one end of the balance, many times this articles fall in the other.
If I tried to deep dive and study as much as is suggested, I'd have no energy for other things that I value more right now. Reading author articles it seems to me others have different energy or their effort is focused differently, but I'm always cautious because burning out it's not something I want to ever repeat
Absolutely agreed. I've had some type of insomnia on and off most of my life, last year was particularly bad. Started taking melatonin, first day I fell asleep and rested like I couldn't for the past year. Asked my doctor about and he told me to keep taking it as long as I felt it worked, since anything he could give would be much much more addictive
Maybe it would be useful what "tell anyone what I read" means. Because if you mean 1 to some in a room, then most likely. If you use any type of broadcasting then most definitely no. Try reading outloud a script from a recent movie on twitch/youtube/radio/tv and whether it gets DMCA'd or not. Same for books, songs I guess... not? But not sure.
This type of comments can be seen every single time a thread about LLM, or OpenAI or some such comes up.
And it adds nothing. I'm sorry but saying "Whether that content is used to train a human mind or an artificial one is probably not up to you" may be worse than saying nothing at all.
First because it shows enough doubt on whether it's up to the authors of content (IP laws, fair use, intent of the use, and many things I ignore), while giving no laws as an example or frame of reference.
And second because it's comparing a human mind that we know exist, to an artificial one, which implies:
1. An LLM is an artificial mind, or close to one, whatever that is (again, not defined).
2. If they were to exist, they would be both equivalent and treated the same as a human one.
The amount of jumps in a couple sentences, added to the uncertainty of how copyright would/will work, multiplied by the numer of times I/we read that type of comment every single time, it's getting tiresome. And it's adding noise to the noise-signal ratio.
Isn't the paper discussing precisely the opposite? That chatgpt predicts text in a way that, with each version, resembles more and more human logic, both with its succeseses and errors?
Quite surprised by others finding this as a... Surprise? I get there is people who never experience this, but they also not know anyone personally to whom this would happen?