Derander · HackerTrans

Derander

no profile record

Submissions

Accelerating Real-Time ML with Symbolic Python Interpreter

chalk.ai

3 pointsby Deranderw zeszłym roku1 comments

Deep Dive: Building PartiQL Support at Chalk

chalk.ai

4 pointsby Derander2 lata temu0 comments

Palantir announces confidential draft S-1

businesswire.com

24 pointsby Derander6 lat temu4 comments

Kademlia: Distributed Hashtable (2002)

maymounkov.org

5 pointsby Derander12 lat temu0 comments

comments

w zeszłym rokudiscuss

Python is the language of choice for many ML teams — it’s flexible, expressive, and has the right ergonomics for real-time workflows. But it’s also slow.

To make Python faster, I built a Symbolic Python Interpreter to make Chalk’s “Python resolvers” into optimized Velox-native expressions — enabling high-performance execution without sacrificing developer experience.

This post explains the problem I encountered, why it’s so challenging, and how we solved for it.

9 lat temudiscuss

That's odd. I work in a 100% typescript shop and about half the devs use intellij and webstorm full time without much trouble.

Jump to definition/auto import/implement interface/etc all work.

9 lat temudiscuss

Arbitrary example: If you're using SHA256 you're emitting 256bit hashes. Hopefully the distribution of SHA256 outputs is indistinguishable from random (or close enough) or else we've got big problems in other domains.

If you're drawing at random from a pool of size k you need approximately sqrt(k) draws until you reach a ~50% chance of a collision[0].

With 256 bits, there are 2^256 possibilities, so following the rule-of-thumb you'd need 2^128 draws until you had a 50% chance of a collision.

2^128 > # of atoms in the universe.

If you adjust your risk tolerance you'll have different numbers come out, but the chance of a collision in any realistic scenario is negligible.

[0]: https://en.wikipedia.org/wiki/Birthday_problem

10 lat temudiscuss

If you're using typescript and emacs, https://github.com/ananthakumaran/tide is a very good integration with the built-in language server for semantic auto-completion and "jump-to" support. I use it every day on large projects.

I think that vscode uses the typescript language server behind-the-scenes for plain javascript analysis.

10 lat temudiscuss

I attempted to use clang-format on a mixed typescript/tsx codebase and found that it mangled jsx expressions. This wasn't surprising but I suspect that typescript users probably overlap heavily with react users so it's not a great solution as is.

10 lat temudiscuss

It's not hard to automate with raw gulp or webpack, but it does add a significant amount of time to compilation. It roughly doubled our incremental compile times. My team decided it wasn't worth it, but it's possible that a cleverer person could decrease the overhead.

10 lat temudiscuss

TypeScript has dealt with this in the past by adapting to the introduction of `<Foo>` from jsx, which clashed with typescript's casting operator. If this ends up being a problem I'm sure that the TypeScript team will just pick a different syntax.

In my experience, the overhead caused by TypeScript is far outweighed by benefits in tooling and confidence that my code works (mostly) after refactors.

11 lat temudiscuss

Lodash has many features that underscore does not have. Also, I am told that Lodash has been implemented in such a way that functions are largely independent of each other so that you can build a custom lodash package that only includes the subset that you use.

11 lat temudiscuss

I TA for this class. The class is optional for non-engineering majors but has enrollment >= 700 people per quarter. So many people at Stanford take the class that it's a bit of a cultural phenomenon.

There are three "intro" programming classes at Stanford. CS106A is "intro to java", CS106B is "intro to C++", CS106X is CS106A + CS106B + a bit more in one quarter. People who already know how to program are encouraged to take one of the latter two, although there is always a contingent of people with /some/ background in 106A.

12 lat temudiscuss

I'm not the OP but:

The analytical gradient is actually a general tool used in multivariable calculus. It's vector-valued, in that if you take the gradient of a function of three variables: f(x, y, z), then you'll get a 3-vector back. Vectors are defined by two characteristics: a direction and a magnitude. The gradient's direction is the direction of greatest increase and the magnitude of the gradient is the instantaneous rate of change.

The gradient is being put to work here in order to optimize a function using a process called gradient ascent. Intuitively it makes sense that in order to optimize your function you'd want to "move" in the direction of greatest increase at every step. However, the size of the step that you take is tricky. As you point out, in this case, we can increase the objective function value more if we double the step size. However, you're not actually doubling /the gradient/.

If you look at the expression that you wrote you should see:

    x + 2 * dx * step, y * 2 * dy * step.

What you've done is double the step-multiplier, not the gradient itself (<dx, dy>). This means that the optimization process jumps further at each step. However, the step size that OP chose is somewhat arbitrary to begin with, so it's not clear why any particular choice would be better or worse. The reason that the step size matters is that if your step size is too large your optimization process might jump over a minimum location or something similarly bad -- there are pathological functions like the Rosenbrock Function [1] which are notoriously hard to optimize with gradient ascent. In practice, you'll often choose your step size more intelligently based on a few different tests, or vary it as the optimization process progresses.

In this particular instance, the surface that you're trying to optimize is pretty simple so basically any step value will do the trick. It may take a different number of steps in order to compute the global maximum, but most reasonable step sizes will get there.

[1] http://en.wikipedia.org/wiki/Rosenbrock_function

12 lat temudiscuss

I've heard nothing but good reviews of the online version of the class. I took the class at Stanford (and actually worked on the system mentioned in the article) and I found its content to be useful. I believe that the online version contains less theory but this is not necessarily a bad thing if all you want is an introduction.

I've also found reading papers to be illuminating: often the first article about a given classifier is fairly well written and accessible if you have a strong background in math.

This is also a useful thing to keep in mind: http://scikit-learn.org/stable/tutorial/machine_learning_map...

12 lat temudiscuss

I'd be surprised if the average high school algebra course covered group theory :-).

12 lat temudiscuss

The standard argument is that people shift jobs so much because it's the most reliable way to get a large pay raise. If companies stopped making it necessary to hop jobs in order to make closer to what you're worth, then it might help retention.

12 lat temudiscuss

One of the interesting things about classic ciphers is that (as far as I know) they're obviously insecure by modern standards.

Most of the time what people want out of modern crypto is something called "semantic security" -- in other words, an attacker should not be able to learn even a single bit of the plaintext for any message, even if they are allowed to see the encryption of polynomially many chosen plaintexts (ignoring length, of course.)

What this means is that deterministic encryption schemes are all broken. Consider a game:

The attacker asks to see the encryption of messages a and b, receiving a' and b' as ciphertexts of a and b.

Now, the attacker is "challenged" with a new ciphertext of a or b, c. If the attacker can (with reasonable probability) distinguish which plaintext c is the encryption of, then the encryption scheme is not "semantically secure."

Notably, since our encryption scheme is deterministic, c = a' or b'. Therefore, with probability = 1.0 the attacker can distinguish which plaintext encrypted to c.

So, by modern security standards all deterministic (and therefore all classic, I think) ciphers are broken.

This is the sort of thing which is covered in the coursera class. Modern crypto is (usually provably) substantially more secure than what used to be used, and the kinds of techniques used to do modern crypto don't have much overlap with things like Caesar ciphers.

12 lat temudiscuss

I'm a current Stanford junior. In my experience most people feel stressed if they haven't committed to an offer by early February. The major career fairs for internships take place mid-January, with a more full-time oriented fair occurring in October.

The majority of CS majors I know were locked in before mid-February.

13 lat temudiscuss

There is probably useful analysis that they can perform in aggregate across all of their users. Perhaps people who read email are more likely to do <x> at a certain time and that signal is only noticeable with more data than you as an individual generate.

13 lat temudiscuss

For those sorts of things I think it's completely reasonable to drop into emacs, vim or sublime and then just copy paste back into your primary IDE.

When it's a text editing operation that you're trying to do (as opposed to a defined transformation on valid source) then it makes sense to use a text editor.

I keep emacs and intellij open usually.

13 lat temudiscuss

I care about cell phone performance because I'm excited for the day when I can realistically start booting a general purpose OS on my phone and call it good enough w/ a monitor and external keyboard.

13 lat temudiscuss

To a certain extent that's what the JVM is -- you just have to target its bytecode :-).

13 lat temudiscuss

One thing to note about this specific problem: this is an example of the traveling salesman problem with a metric. This makes efficient (good) approximation algorithms a possibility.

Many NP-hard approximation algorithms classes teach a 1.5 approx known as the Christofides algorithm. This algorithm is guaranteed to provide an approximate solution that is no worse than 1.5 times the optimal total distance, and often much better.