abuckenheimer · HackerTrans

abuckenheimer

no profile record

Submissions

Google play music shutting down

blog.youtube

6 pointsby abuckenheimer6 anni fa1 comments

IEX exposes free web api

iextrading.com

2 pointsby abuckenheimer9 anni fa0 comments

comments

anno scorsodiscuss

excellent explanation, to add to this since I was curious about the composition, '%c' is an integer presentation type that tells python to format numbers as their corresponding unicode characters[1] so

'%c' * (length_of_string_to_format) % (number, number, ..., length_of_string_to_format_numbers_later)

is the expression being evaluated here after you collapse all of the 1s + math formatting each number in the tuple as a unicode char for each '%c' escape in the string corresponding to its place in the tuple.

[1] https://docs.python.org/3/library/string.html#format-specifi...

2 anni fadiscuss

This is really nice! The context switching that comes from using the [official k8s reference](https://kubernetes.io/docs/reference/kubernetes-api/workload...) is a real pain. If your writing a deployment and need to check one thing about the pod spec all of a sudden you jump to a new page and lost where you were. Ontop of that this keeps track of the indentation level the spec your looking at within the context of whatever parent path your writing it for.

Maybe one nitpick would be to keep colon between the key and the type so one can copy paste multiple lines of relevant spec to be filled in your editor easily.

3 anni fadiscuss

I've only skimmed the quickstart docs https://kargo.akuity.io/quickstart/ but this snippet immediately clicks for me as something that I'd want to try

    cat <<EOF | kubectl apply -f -
    apiVersion: kargo.akuity.io/v1alpha1
    kind: Promotion
    metadata:
      name: prod-to-${FREIGHT_ID}
      namespace: kargo-demo
    spec:
      stage: prod
      freight: ${FREIGHT_ID}
    EOF

Have enjoyed using argo in general in the past, its got a great model for k8s native workflows / events but never got to using it for CD.

4 anni fadiscuss

This is clever, otherwise you have to coordinate your cache setter and your async function call between potentially many concurrent calls. There are ways to do this coordination though, one implementation I've borrowed in practice in python is modes stampede decorator: https://github.com/ask/mode/blob/master/mode/utils/futures.p...

4 anni fadiscuss

> python3 -c 'while (1): print (1, end="")' | pv > /dev/null

python actually buffers its writes with print only flushing to stdout occasionally, you may want to try:

    python3 -c 'while (1): print (1, end="", flush=True)' | pv > /dev/null

which I find goes much slower (550Kib/s)

6 anni fadiscuss

Very cool, py-spy[1] has been an invaluable tool in my development process since jvns blogged[2] about it. The power of being able to visualize where your code is spending its time is so obvious and I'm glad people are building tools to make that easier.

As a quick compare and contrast between py-spy and pyinstrument it looks like py-spy has the advantage of being able to attach to an already running process which is super useful when your program is stuck and you don't know why. I haven't used pyinstrument yet but I do like the fact that it can do its flame graph in the console, sometimes I find saving down an svg file and opening up the browser a bit arduous. Excited to give it a try.

[1] https://github.com/benfred/py-spy

[2] https://jvns.ca/blog/2018/09/08/an-awesome-new-python-profil...

6 anni fadiscuss

It's interesting to compare palantir's risk factor on open source vs pivotals[1]. Palantir takes a defensive posture saying basically that their ability to not contribute to open source without running afoul their licenses is important:

Our platforms contain “open source” software, and any failure to comply with the terms of one or more of these open source licenses could negatively affect our business.

Where as pivotal says that being an active part of a vibrant open source community is crucial:

If open-source software programmers, many of whom we do not employ, or our own internal programmers do not continue to use, contribute to and enhance the open-source technologies that we rely on, the market appeal of our offering may be reduced, which could harm our reputation, diminish our brand and result in decreased revenue.

Like this makes sense, palantir is a closed source first kinda company and obviously a different value to its customers than pivotal. Kinda interesting to see that manifestation in the risk factors though.

[1] https://www.sec.gov/Archives/edgar/data/1574135/000104746918...

7 anni fadiscuss

Just remembered the puck building[1] has these on the sidewalks surrounding the Houston street side, if you go into the Eastern Mountain Sports store and down the stairs you can see them from below.

[1] https://en.wikipedia.org/wiki/Puck_Building

7 anni fadiscuss

Would highly suggest listening to the "How I built This" podcast[1] with the peleton founder John Foley, really great story with usual tech themes like doing things that don't scale and finding a niche but loyal customer base and expanding organically.

[1] https://play.google.com/music/m/Dmjr6m5i4h3j6gaysgjd47ryhci?...

8 anni fadiscuss

Yes HFT buys trade flow from robin hood because they make more money executing against it but that's not actually to the detriment of the people on the robin hood app. The main way HFT firms make money is by making a market, they offer to buy and sell stocks cheaper than anyone else and get paid by people crossing the spread and sometimes exchange fees. The reason robinhood trade flow is valuable to HFT firms Isn't because they are trading against "dumb" millennials but because they know millennials aren't likely to move the market playing around on their smartphone. HFT firms can collect a small rent sitting in between millenials trading with one another without the risk of being on the wrong side of a trade that materially moves the price of a stock.

Matt Levine does great write ups on this stuff, would highly recommend: https://www.bloomberg.com/opinion/articles/2018-10-16/carl-i...

8 anni fadiscuss

got it, thanks for the response!

8 anni fadiscuss

Right what I'm confused about is that first bit, my understanding from the RFC is that the implementation should have look something like

    return pbkdf2.derive(password, email, PBKDF2_ROUNDS, STRETCHED_PASS_LENGTH_BYTES)
      .then((quickStretchedPW) => {
        result.quickStretchedPW = quickStretchedPW;
        // stretch to twice the length necessary
        return hkdf(quickStretchedPW, kw('generated'), HKDF_SALT, HKDF_LENGTH * 2)
          .then((generated) => {
            // split output into two cryptographically strong keys
            result.unwrapBkey = generated.slice(0, HKDF_LENGTH);
            result.authPW = generated.slice(HKDF_LENGTH);
          }
        );
      }
    )

but my read in pseudo code of what they end up doing is closer to this:

    hashed_password = hash(password, 'salt1')
    hashed_auth_tok = hash(hashed_password, 'salt2')
    hashed_unwrap_key = hash(hashed_password, 'salt3')

which seems secure because the server can't reverse hashed_unwrap_key to find hashed_password and thus shouldn't be able to calculate hashed_auth_tok. However the point of HKDF is to make multiple cryptographic keys while it looks like in practice we are just using it as a one way funciton.

8 anni fadiscuss

I've never heard of HKDF before but it is really an elegant solution to this. My first guess on how to do this would have been something stupid like split the Authentication token in half and 0 pad it. But this would have significantly reduced the entropy available on both keys, reducing the search space on the authentication token and the encryption key making them much more brute force able. HKDF instead expands the key and essentially requires the server to be able to reverse HMAC-Hash to find the encryption key from the the authentication token.

What I'm confused about is that they seem to be using HKDF as a hash [1] and not as a key generation funciton. I think this is just as secure as what I was expecting but it seems more complicated and doesn't jive with the purpose of the RCA[2] as I read it.

[1] https://github.com/mozilla/fxa-js-client/blob/1d92f0ec458ace... (separate HKDF calls with the same IKM)

[2] https://tools.ietf.org/html/rfc5869

8 anni fadiscuss

I loved this article on the elegance of deflate http://www.codersnotes.com/notes/elegance-of-deflate/ finishes with this:

It's not one algorithm but two, that dance and weave together in harmony. You can pick dictionary matching, or entropy coding, and you can select between them on a per-byte basis with no overhead. That's what I find so clever about it - not that it can do one thing or the other, but that it can choose either, and yet represent them using the same language. No extra markers, nothing to say "oh, now we're switching to entropy mode", nothing to get in the way.

8 anni fadiscuss

[disclosure I am a sanic contributor] aiohttp is a great project, very similar to sanic, I'd say the big thing is sanic is slightly more flask like than aiohttp in ergonomics (which is something aiohttp has been working on[1]).

Sanic also makes performance a key development goal so you can feel comfortable the project will scale gracefully. But I will stress I don't have current benchmarks to compare against aiohttp

[1]https://aiohttp.readthedocs.io/en/stable/faq.html#id1

8 anni fadiscuss

Really I think the specific the problem here with DAF's is just misaligned incentives, the DAF values the donors donation but the government is the one that has to pony up the tax rebate. Obviously there is some auditing mechanism here where the valuation can't be too far from reality otherwise the government would come down on it but in practice it seems like that is one of their benefits. I think the more interesting and subtle problem though is that the rebate is awarded at donation time instead of when the money actually gets put to use which lets the DAF accrue fees on government money.

in a highly stylized example:

1. A donor gives the DAF a $10m yacht

* DAF gives a donor $10m donation receipt

* Government gives donor <donor tax rate> * <donation receipt> rebate i.e. .30 * $10m = $3m

2. DAF liquidates yacht into $9m cash and invests it

* the DAF charges .006 management fee on this for 5 years with a 5% return

3. At the end of 5 years the DAF liquidates its investments and puts all of that money to work on a cause.

* ~$11.2m goes to work on a charitable cause

* DAF has collected ~340k in fees

This is more or less the description the DAF in the article but now but we've simplified to assume a conservative consistent return and that the whole donation gets put to work at once at the end of a set term whereas more likely it would be slowly liquidated over time. Now let's assume that instead of the government paying up the tax rebate when the donation is made that it is paid out when the money left the DAF and went to work on a cause.

1. donor gives a DAF $10m yacht

* DAF values it at $9m and gives the donor <donor tax rate> * <donation receipt> rebate i.e. .30 * $9m = $2.7m

2. DAF liquidates yacht into $9m cash, covers their $2.7m rebate outlay and invests $6.3m

* the DAF charges .006 management fee on this for 5 years with averages a 5% return

3. At the end of 5 years the DAF liquidates its investment and puts all of that money to work on a cause.

* ~$7.8m from the DAF goes to work on a charitable cause + a $2.7m from the government to match that original rebate for a total of ~$10.5m

* DAF has collected ~238k in fees

What's the difference here?

1. The government saves $300k on a rebate because the DAF has to liquidated the yacht to come up with the money to pay the rebate so the valuation is based on something the reflects the actual cash

2. The government keeps <rebate amount> ($2.7m in this case) in its bank account for the 5 years while is in the DAF. This means the government can use it for it's own operations over that time period rather than it sitting in the "warehouses of wealth". It also means the DAF can't collect fees on this money because it is not managing it.

I'm not trying to make a statement on if the government should or should not subsidize donations to charity, just point out how that subsidy seems to be leveraged by the introduction of DAFs against its original intent.

sorry for the terrible formatting

8 anni fadiscuss

I don't know the Kirill Balunov example is pretty beautiful/simple/flat/readable

    if reductor := dispatch_table.get(cls):
        rv = reductor(x)
    elif reductor := getattr(x, "__reduce_ex__", None):
        rv = reductor(4)
    elif reductor := getattr(x, "__reduce__", None):
        rv = reductor()
    else:
        raise Error("un(shallow)copyable object of type %s" % cls)

especially when you compare it to the existing implementation:

    reductor = dispatch_table.get(cls)
    if reductor:
        rv = reductor(x)
    else:
        reductor = getattr(x, "__reduce_ex__", None)
        if reductor:
            rv = reductor(4)
        else:
            reductor = getattr(x, "__reduce__", None)
            if reductor:
                rv = reductor()
            else:
                raise Error("un(shallow)copyable object of type %s" % cls)

8 anni fadiscuss

I think the reality of this is fairly well captured in the 4th to last paragraph

> Even human readers, who may have two minutes to read each essay, would not take the time to fact check those kind of details, he says. "But if the goal of the assessment is to test whether you are a good English writer, then the facts are secondary."

I would go a step further and say the goal of standardized tests is probably not to test if your a good English writer but rather to see if your good at that test. AI has tremendous ability to streamline the grading of standardized tests which are narrowly focused measure and I think that's fine if it comes with proper checks. As many people mention here I don't think AI will be able to objectively measure good general writing and I doubt we'd fully cede artistic evaluation to a computer as a society but I agree with being weary of this.

Standardized tests in general though seem to me like one of the greatest examples of a measure becoming a target and loosing a lot of its effectiveness (Goodhart's law). I can't help but be reminded of Paul Grahm's essay on nerds[1] by this article because the effective point of school from an evaluation standpoint is these tests. However now we've built a computer that grades these test's nearly as well as its human counter parts and its glaringly obvious that this evaluation is tremendously game-able. Which resonates with Paul's argument that: "The problem with most schools is, they have no purpose."

[1] http://paulgraham.com/nerds.html

8 anni fadiscuss

I feel the same way, although my instinct is generally to build a custom generator. Only costs a couple lines but is plain old python and quite explicit

    target = {'system': {'planets': [{'name': 'earth', 'moons': 1},
                                     {'name': 'jupiter', 'moons': 69}]}}

    glom(target, {'moon_count': ('system.planets', ['moons'], sum)})
    # vs
    def iter_moons(t):
        for planet in target['system']['planets']:
            yield planet['moons']

    sum(iter_moons(target))

would have to combine with `defaultdict`s if your nested data is only sometimes there though

8 anni fadiscuss

FWIW no one who replied to this email thread said something even close to "no". Victor Stinner points out that startup time is something that comes up a lot and mentions some recent work in the area [1].

Python is a big ship, it may not be as nimble as a young FOSS project but it is always improving and investments in things like start up time pays dividends to a large ecosystem.

[1] https://mail.python.org/pipermail/python-dev/2018-May/153300...