stack_underflow · HackerTrans

stack_underflow

no profile record

comments

stack_underflow

letztes Jahrdiscuss

It sounds like they're doing something similar to Seagate's Field Access Reliability Metrics (FARM) log where IIRC it's much harder to reset or forge their wear-leveling/usage stats, vs SMART metrics which certain manufacturers seem to clear when drives are re-certified[0]. I've seen this tool[1] mentioned often in /r/DataHoarder discussions about checking whether second hand drives have had this stat-reset done. I'm assuming it compares `smartctl --log=farm` output with the attribute/device-statistic log counter values.

A friend and I have been building our own solution[2] for monitoring these wear-leveling attributes on NVMe and SATA drives, with the focus being on tracking and visualizing trends over time. We both have a large collection of drives in various servers and laptops and found that SMART metrics can be reported somewhat inconsistently from vendor to vendor so what started as a simple shell script to scrape `smartctl` output has now turned into a lightweight desktop agent that attempts to normalize all these inconsistencies and let us focus on the actual signals while also allowing us to define alerts/notify us of anomalies via email - maybe something HN users will find useful.

Fun fact: did you know that most drives maintain a pool of spare sectors/cells that are used by the firmware to replace blocks that have failed? It's one of the many metrics we like to track and visualize in Sentinowl [2]!

[0] https://github.com/gamestailer94/farm-check/tree/main

[1] https://www.heise.de/en/news/Fraud-with-Seagate-hard-disks-D...

[2] https://sentinowl.com/

stack_underflow

vor 3 Jahrendiscuss

Just curious, any reason you're using TMarkor instead of Markor? (https://github.com/gsantner/markor/ , https://play.google.com/store/apps/details?id=net.gsantner.m...)

Only asking since this was also the editor I ended up settling on in Android, but it seems like TMarkor is just a repackaging of Markor without any references to its forked(?) source.

My requirement was that the editor had to be open source so that I could audit the repo and compile the APK from source, as well as potentially fork it for personal modifications if needed.

stack_underflow

vor 3 Jahrendiscuss

Can also vouch for this. Bought a 2nd gen AMD Thinkpad T14 ~last year and haven't had any issues at all running Debian testing (bookworm). Swapped the m.2 with a 4TB one and added an extra 16GB of RAM to supplement the soldered 16GB. I believe they're on 3rd gen now but I imagine hardware compatibility is probably similar to 2nd gen.

The size/form factor is basically the same as my old T460s which I've had for ~8+ years and is still running, but a bit dated now with a 2c/4t Intel CPU. I believe back then the T460s was the "slim" version of the T460, but Lenovo seems to have dropped the "s" for the T14 even though the dimensions of both seem the same to me, maybe a couple extra mm in height on the T14.

stack_underflow

vor 3 Jahrendiscuss

One of my quick-and-dirty gotos for getting a rough idea of buffered-writes size + disk-write activity on random linux systems is: `watch -n1 grep -ie dirty -e writeback /proc/meminfo`.

You can invoke `sync` to watch the buffered-writes queue burn down when you have lots of pending writes.

see: `LESS=+/meminfo man proc` or https://github.com/torvalds/linux/blob/master/Documentation/... for more info

stack_underflow

vor 3 Jahrendiscuss

I think anyone who suggests daily driving Debian testing should also mention the fact that packages can disappear from testing for weeks/months at a time (and reappear later). It's recommended to configure `unstable` in your sources as well but set up apt pinning so that those packages are only pulled in if they're missing in testing. See: https://wiki.debian.org/DebianTesting "Best practices for Testing users"

In practice this means adding something like this to /etc/apt/preferences (along with adding entries for `unstable` in /etc/apt/sources.list)

    # use `n=` when referencing codename (i.e. buster/bullseye/...)
    Package: *
    Pin: release n=bookworm
    Pin-Priority: 550

    # use `a=` when referencing archive (i.e. stable/testing/unstable)
    Package: *
    Pin: release a=unstable
    Pin-Priority: 520

That way apt will pull in any packages missing in testing from unstable, and once the package is reintroduced to testing, will prefer that version rather than continue to track unstable.

Maybe I've been lucky but I've been running testing on my non-server desktops and laptops for 13 years now and have only rendered my system unbootable once (required having to boot up a live CD to reinstall an older working version of some bad libpcre update that had been rolled out).

stack_underflow

vor 4 Jahrendiscuss

If you're asking about specific examples, this book covers a lot of common patterns I've seen over the years; maybe a good starting point:

https://martinfowler.com/books/refactoringDatabases.html https://databaserefactoring.com/

stack_underflow

vor 4 Jahrendiscuss

> This eventually succeeded, giving the threat actor access to one of our GitHub organizations where they proceeded to copy 130 of our code repositories.

When I was at dropbox I saw quite a few devs provisioning non-mac laptops with their linux distro of choice. Always wondered what was stopping those people from just making a copy of rserver/rclient without IT/security noticing.

stack_underflow

vor 5 Jahrendiscuss

The Kurose/Ross textbook is also a favourite of mine.

Consider checking out https://book.systemsapproach.org/index.html as a free alternative as well.

The authors develop the book, as well as a few micro-books on topics like SDN, 5G, TCP congestion control, on github: https://github.com/SystemsApproach

They also have a blog with regular updates: https://www.systemsapproach.org/blog

And if you're curious about the motivation behind the project, they have a whitepaper: Open Source Networking Education: A Systems Approach - https://www.systemsapproach.org/uploads/4/7/7/4/47748257/sig...

stack_underflow

vor 5 Jahrendiscuss

Sure, but there are also strategies to buffer yourself from that. Barring the first year where you have to wait/make it up to the 12-month mark to get the full 25% of your equity paid out, 99% of companies (i.e. almost everyone except for Amazon IME) will vest 1/4 of your annual 25% stock comp quarterly - I've even seen 1/12th-monthly in an offer.

If you're willing to take the short-term cap gains hit you can sell your stock immediately after vesting and reinvest in a total market index or w/e aligns with your investing philosophy. If we're talking hard numbers, you'd still be pulling in 150-200k+ in base salary/cash and let's say, in a non-ideal situation, what would've been your $125k of annual stock vest is now worth 30% less - still not a bad deal IMO.

stack_underflow

vor 5 Jahrendiscuss

One thing I've noticed is that the people making these counter-arguments seem to completely ignore (or don't know about) the ridiculous amounts of equity that $big_tech_co's are handing out.

Yes some of those points can be valid, and I understand that not all software companies hand out that much stock or are private and therefore it's harder to depend on, etc. But maybe it's just the bubble of tech that I've existed in within the PNW, but tonnes of people in my circle have built FIRE-level wealth by just having been driven enough to put up with bullshit interviews and staying long enough to get their stock grants.

I can say personally I definitely wouldn't have made it to where I did financially had I stayed in Canada, or it probably would've taken me at least 15 years instead of ~5.

If you're the type of person for who money can solve a lot of problems, I always suggest considering this as an option. It's solved ~90% of the problems in my life and has bought me years of time to be able to do what I actually want in life.

stack_underflow

vor 5 Jahrendiscuss

Your post reminded me of the presentation on cantor.dust:

  https://sites.google.com/site/xxcantorxdustxx/

  https://www.youtube.com/watch?v=4bM3Gut1hIk - Christopher Domas The future of RE Dynamic Binary Visualization
    (very interesting presentation)

Looks like there's even been a recently open sourced plugin for Ghidra released by Battelle:

  https://github.com/Battelle/cantordust

stack_underflow

vor 5 Jahrendiscuss

Seems like I struck a nerve here... I didn't realize I'm not allowed to use HN to start discussions with other readers about topics they bring up :)

I like to post about these kinds of thoughts on HN because I feel the crowd is the type to offer useful insights I don't find elsewhere on the internet or even IRL. Do I have a tendency to ramble? Sure, look at my post history...

If you have any insights about what I mentioned (a thing I've been curious about for a long time), I'm all ears - but I don't see how being salty is contributing.

stack_underflow

vor 5 Jahrendiscuss

That's a really good point. I've also done what you mentioned in your first paragraph but stopped digging deeper once I hit the "now I have to understand physics to actually understand why electricity behaves the way it does in order to flow through transistors/logic gates" barrier. My reasoning at the time was a combination of asking myself how much time I actually had to dive into that as well, plus how much of a potential return I thought I could get from having resolved those questions.

stack_underflow

vor 5 Jahrendiscuss

I feel like I'm on one extreme of this spectrum which is: I 'need' to develop a deep understanding of systems from the bottom-up and only then can I feel comfortable building on top of it/abstracting it away/being productive with it.

When I started learning socket programming I didn't feel completely comfortable with using it until I had a complete understanding of the networking stack from the PHY upward. I even needed to write my own simple ARQ implementation to feel like I could fully appreciate and have a "feel" for what two machines are "actually doing" when they're sending packets back and forth.

I had the same obsessive desire to completely understand from the bottom-up how database engines were implemented because I just didn't feel comfortable with just the knowledge that "selects on tables with indexes on the field being queried are 'fast'". I just _had_ to know what was making them fast. Funny thing, when I found out that databases and database indexes were just disk-persisted/in-memory self balancing search trees or hash tables, I thought to myself, "that's it? why didn't anyone just tell me that instead of just repeating "queries are fast/optimized by the db"?"

I don't know if my brain just works differently because it seems most people I've worked with seem to get frustrated/annoyed with these details and not even have any desire to learn a system in-depth like this. They seem to only want to reason about a system at its highest level of abstraction, whereas to me, with the database example, once I learned that B+ trees were one of the primary data structures that backed database systems, I felt like the system as a whole just became that much more transparent and easy to reason about because now I could leverage all of my intuitions+past experiences relating to performance of operations on tree structures, having an intuition of perf implications of using linked data structures, reasoning about issues like paging chunks of data in and out of disk, etc.

stack_underflow

vor 6 Jahrendiscuss

I really like that persistent link-background highlight effect when you click a link.

stack_underflow

vor 6 Jahrendiscuss

Somewhat related; I've always been very curious why a lot of developers I've worked with seem to think testing isn't worth the effort, and I've been keeping a mental checklist of all the reasons/excuses I hear so that I can reflect on them as well against my own experiences - sort of a way to challenge myself and ensure I'm not just cargo-culting methodologies.

Recently I decided to go through `Growing Object-Oriented Software - Freeman, Pryce` (and actually finish working through it this time) with the goal of understanding what "proper TDD" is supposed to look like. Something interesting I noticed is almost all the complaints I've seen as reasons against TDD/certain testing methods all seem to be examples the authors use as how not to use TDD/testing. It seems to me that a lot of developers have just learned _of_ these techniques by name, and haven't really put a lot of time/effort into practicing their application and instead just seem to write them off on face value or by the literal interpretation of their names.

One example, a lot of people I speak with seem to think TDD is very literal "write a _unit-test_* for everything before you write the implementation OR ELSE...", but of course there's a lot more nuance than that depending on the situation. The book actually puts a huge emphasis on having your initial test(s) be end-to-end tests that slice through the system as a whole, with the idea being to create nested feedback loops of varying granularity/abstraction to allow you to iterate without fear. This was something I never heard emphasized at all when I started learning about TDD, or hell even in a paid course my employer put us through by a "TDD expert".

I should also clarify, I only consider these claims from developers that have a proven track record of working on large/complex systems since, well, those who don't probably haven't cultivated their ideal workflows/approaches yet (or just haven't been given the opportunity to showcase them, as is common in large companies).

* I emphasize unit-test because a lot of code bases I've worked in rarely have anything other than just unit-tests...

stack_underflow

vor 6 Jahrendiscuss

I had to investigate why our docker-ized dev environment was so slow on macOS at my last job and this was the root cause. One project's test suite ran in about 5-10 mins in our linux CI/CD environment, and about 50+ mins in macOS with docker.

There was a very in-depth thread on the docker forums where the devs explained why there was such a huge performance penalty. IIRC it was due to all the extra bookkeeping that had to be done to ensure strong consistency and correct propagation of file system events between the virtualized docker for mac environment and the host file system.

The test suite would run integration tests that performed a lot of npm/yarn operations which meant lots of disk IO.

stack_underflow

vor 6 Jahrendiscuss

Curious, does one need to use nixOS for an optimal experience?

I've been using debian for a long time and would prefer not to have to switch OSs but the idea of using nix for having full control over my package graph is very tempting.

I've been somewhat procrastinating on trying nix as I've heard GNU Guix has a similar feature set and haven't been able to decide on which one to dive into...

My ideal setup would be to just be able to run a single shell script that configures a new machine to the exact state of all my other dev machines. I have a shell script that somewhat does this but it's not completely unattended and still requires a lot of manual config for certain steps.

Aside from that my main use case is being able to easily share a dev/build environment with others for ensuring that they can compile a certain project exactly as I do. For now I just use docker but it's frustrating not having explicit control over the layer cache and being able to tell it what to cache and what not to.

stack_underflow

vor 6 Jahrendiscuss

Sounds to me like you're focusing on the wrong things or don't have the right resources for the task at hand. (that is, I don't think I've ever had to brush up on a proof for any algorithm or data structure when prepping for interviews)

Take it from someone who also had a shit background/education in math/CS and had to teach myself a lot of these things from scratch - it's doable, you just need time, and I guess the mindset that you can learn these things. See my comment history for some book recommendations. If you really feel like you need to learn things from scratch to give yourself the confidence of having a solid foundation, look at some of the intro computer science courses on Coursera. When I realized most of my uni's cs courses were... not great, I just ended up going through courses like Stanford's Introduction to Algorithms on Coursera and teaching myself.

I can also tell you as someone who was in Seattle for a while and had a bunch of friends at Amazon, I don't think anyone is going to think less of you, if anything, being able to survive at a place like Amazon seemed to be recognized as its own accomplishment...

edit: I'll also add (from personal experience), if it still feels like these things are difficult even once you feel like you have all the right resources, look into seeing medical professionals to make sure you're not being held back by anything, like sleep apnea, any cognitive disability (not sure if that's the right term...) etc

stack_underflow

vor 6 Jahrendiscuss

This has been my take on it as well. Sure it sucks having to jump through these hoops and sometimes even being evaluated by people you know don't understand the topic as well as you and are looking for specific answers/solutions, but this same process also helped me, a "C/D student" from a no-name school and a poor family with 0 connections in the tech industry end up with job offers in the $300k+ range.

The only thing I'm salty about is that I can only get these offers in the US and not in Canada and that my options for permanent residence in the US seem to be gated behind an 80+ year wait... https://medium.com/@happy_sushi_roll/the-endless-wait-for-a-...