hazrmard

Does the peak starting in mid-2020 correspond to mass speculative / adversarial / bandwagon hiring by tech companies during the pandemic?

hazrmard

12 dni temudiscuss

Looks fun, but I don't quite understand this product:

  - Do the buttons map to configurable skills / prompts?
  - Is it meant to be used remotely with some independence (like codex remote), or is it a peripheral like a trackpad?

hazrmard

23 dni temudiscuss

You'd be right to be surprised by lack of mention of quaternions. I (blog author) was not too familiar with them when I did the work this post is based on. With the benefit of hindsight, I may yet revise the post.

hazrmard

23 dni temudiscuss

Hello, blog author here. I agree the math appears overkill. I wrote this as a learning aid for myself with the benefit of hindsight of having worked with some drone sims. I wanted to dispel any doubt in my mind that I could derive drone physics from basic principles, instead of copy-pasting state equations. I went further and tried to motivate transport theorem, rotational analog of F=ma etc from scratch.

In summary: I take F=ma and extend it for rotational motion. (1) Calculating linear motion when the vehicle containing sensors is rotating. (2) Calculating rotation of the vehicle itself due to thrust/yaw force acting about its center of mass.

I'll echo what the other commenter said: this is no way PhD math. It may appear so - but I'm only being verbose with simpler concepts like cross products and rotation matrices.

hazrmard

23 dni temudiscuss

It's my blog. Bartosz Ciechanowski's work was my inspiration!

hazrmard

25 dni temudiscuss

Good work! I wonder if meta-learning can play a better role here compared to heuristics or hindsight. MAML requires hessians, but first-order MAML or Reptile variants could help apply layer-wise adjustments to learning rates.

hazrmard

26 dni temudiscuss

These day's I'm super-into information theory and entropy, so I liked the connection made at the end. I'm a visual person, and I found this schematic of SVD on the wikipedia page very insightful [1].

[1]: https://en.wikipedia.org/wiki/Singular_value_decomposition#/...

hazrmard

27 dni temudiscuss

This is very impressive! I researched fault-tolerant octorotor control using RL in grad school for a NASA project. Perhaps this may be helpful[1, see section 8.3]! The field is moving fast, so there may be better or more suitable approaches out there now.

For folks who are interested in UAV physics, I wrote up an explainer[2].

[1]: https://drive.google.com/file/d/1RTEVRd0XCWLuDXY2nkbmYuOaa5x...

[2]: https://iahmed.me/post/drone-physics/

hazrmard

3 miesiące temudiscuss

Check my understanding & follow-up Qs:

An auto-encoder is trained on [activation] -AV-> [text] -AR-> [activation], where [activation] belongs to one layer in the LLM model M.

Architecture.:

    Model being analyzed (M): >|||||>  
    Auto-Verbalizer (AV) same as M, with tokens for activation: >|||||>  
    Auto-Reconstructor (AR) truncated up to the layer being analyzed: ||>

The AV, AR models are initialized using supervised learning on a summarization task. The assumption being that model thoughts are similar to context summary.

The AR is trained on a simple reconstruction loss.

The AV is trained using an RL objective of reconstruction loss with a KL penalty to keep the verbalizations similar to the initial weights (to maintain linguistic fluency).

- Authors acknowledge, and expect, confabulations in verbalizations: factually incorrect or unsubstantiated statements. But, the internal thought we seek is itself, by definition, unsubstantiated. How can we tell if it is not duplicitous?

- They test this on a layer 2/3 deep into the models. I wonder how shallow and deep abstractions affect thought verbalization?

hazrmard

4 miesiące temudiscuss

Very cool! Interesting how the choice of solver affects the solution. Euler doesn't handle misbehaved equations very well. You can see this in the Helix setup where the bodies just fly off.

hazrmard

5 miesięcy temudiscuss

I should read up on Tailscale more. I have been using ddclient[1] or the router's built-in dynamic DNS[2] to set up my servers / homelab. This leaves the endpoints exposed to the public internet, as the author says.

    [1]: https://github.com/ddclient/ddclient  
    [2]: https://kb.netgear.com/1058/What-is-Dynamic-DNS-DDNS

hazrmard

6 miesięcy temudiscuss

cue the bell curve meme for learning autonomy:

                 ____.----.____
          ______/              \______
    _____/                            \_____
    ________________________________________

    (simulations)  (real world data)  (simulations)

Seems like it, no?

We started with physics-based simulators for training policies. Then put them in the real world using modular perception/prediction/planning systems. Once enough data was collected, we went back to making simulators. This time, they're physics "informed" deep learning models.

hazrmard

6 miesięcy temudiscuss

    https://iahmed.me

Hugo website, with a theme I made from scratch myself.

Github Pages deployment.

Here's my first website from when I was in college and had no experience in web dev. I still keep it on for nostalgia:

    https://iahmed.me/old_www/

hazrmard

7 miesięcy temudiscuss

This reflects my experience. Yet, I feel that getting reliability out of LLM calls with a while-loop harness is elusive.

For example

- how can I reliably have a decision block to end the loop (or keep it running)?

- how can I reliably call tools with the right schema?

- how can I reliably summarize context / excise noise from the conversation?

Perhaps, as the models get better, they'll approach some threshold where my worries just go away. However, I can't quantify that threshold myself and that leaves a cloud of uncertainty hanging over any agentic loops I build.

Perhaps I should accept that it's a feature and not a bug? :)

hazrmard

7 miesięcy temudiscuss

The paper finds:

- For LLM-assisted output, the more complex the LLM-writing is, the less likely the paper is to be published. From eyeballing, at WC=-30, both have similar chances of publication (~46%). At the upper range of WC=25, LLM-assisted papers are ~17% less likely to be published.

- LLM-assisted authors produced more preprints (+36%).

I wonder:

- What is the distribution of writing complexity?

  * Does the 17% publication deficit at WC=25 correspond to 17% of the 36% excess LLM-assisted papers being WC=25, thus nullifying the effect? Although, it puts extra strain on the review process.

hazrmard

7 miesięcy temudiscuss

I can vouch for this with my experience.

Back in grad school, I was out making new friends. I was playing tennis 4-5 times a week. I'd invite players for post-game coffees (in the morning) and dinner (evenings) at every game. Consistency mattered. I'd ask every time. Slowly we had our regulars. Our coffee times became an institution in and of themselves.

People are busy, yes. But, people also want to be in demand. People also don't want to be rejected. And, people also don't want to be left out.

Asking around, I was exposing myself to rejection. Some folks appreciated their time being demanded. More still joined because they didn't want to be left out.

hazrmard

10 miesięcy temudiscuss

Do I understand this right?

A light-weight speculative model adapts to usage, keeping the acceptance rate for the static heavy-weight model within acceptable bounds.

Do they adapt with LoRAs?

hazrmard

11 miesięcy temudiscuss

This takes me down a memory lane! For my undergrad capstone project, we made a cubesat tracker for our university's satellite using a RPi/Arduino/Software-defined-radio to receive transmissions every time it passed over us. I cringe a little looking at the code now - but it worked!

I agree, cubsats are a wonderful way, for college students even, to tinker with space(-adjacent) tech.

https://github.com/hazrmard/SatTrack

hazrmard

w zeszłym rokudiscuss

I am working on a budgeting app!

Features:

  - Local. No internet connection needed.  
  - Manual. Every transaction is added by the user.
  - One-off or arbitrarily recurring transactions.  
  - No lock-in. Check out your data any time.  
  - Arbitrary metrics to track performance. 
  - Hosting on the cloud for mobile access.

Why?

I've been using Google sheets + forms for the last 8 years to track my finances. It's worked well, except for minor inconveniences. This app is my answer to my own problems.

hazrmard

w zeszłym rokudiscuss

Thank you very much! I had been reading up on effect of diet for treatment & management. I like the focus on citations.

Submissions

Drone Physics

A Derivation of Entropy

Ask HN: What is the latest on treatment of Metastatic Breast Cancer?

GenAI and my six degrees of separation from tensor products

Running Python on air-gapped systems

Show HN: I wrote a multicopter simulation library in Python

Helping fix aircraft – from NLP to Bayes Nets

The epic quest to build a permanent Moon base

Ask HN: How to make inroads into quantitative finance as an engineering PhD?

Ask HN: With recent layoffs, how would you advise new grads entering the market?

Ask HN: How to keep “hard” skills as an electrical engineer working in software?

HTTP Cats

comments