richhickey

7 เดือนที่ผ่านมาdiscuss

Time to update your dictionary -

slop: "digital content of low quality that is produced usually in quantity by means of artificial intelligence"

https://www.merriam-webster.com/dictionary/slop

It is strictly this meaning I intended.

richhickey

ปีที่แล้วdiscuss

Well, at least I know how to form an argument that is not ad hominem. Remind me who you are next time 'KingMob' so I don't waste my time chatting with you again.

richhickey

2 ปีที่แล้วdiscuss

An advantage of using lists is that tx data tends to be built up serially in code. Having to look at your tx data in a different (set) order would make proofreading alongside the code more difficult.

richhickey

6 ปีที่แล้วdiscuss

Companies still compete to attract and retain top devs. What if those devs insisted that the companies they work for sponsor open source, and prefer sponsoring companies when making decisions about where they'll work? If companies spent just somewhere near 1% of what they spend on devs on open source sponsorship they could transform the ecosystem. As a result they'd have more and better open source to leverage. That's before you consider the risk mitigation of ensuring stable suppliers.

richhickey

10 ปีที่แล้วdiscuss

"datum" means "a thing given" - a fact or presumed fact.

http://www.dictionary.com/browse/datum

richhickey

10 ปีที่แล้วdiscuss

If we can't agree on what words mean we can't communicate. This discussion is undermined by differing meanings for "data", to no purpose. You can of course instead send me a program that (better?) explains yourself, but I don't trust you enough to run it :)

The defining aspect of data is that it reflects a recording of some facts/observations of the universe at some point in time (this is what 'data' means, and meant long before programmers existed and started applying it to any random updatable bits they put on disk). A second critical aspect of data is that it doesn't and can't do anything, i.e. have effects. A third aspect is that it does not change. That static nature is essential, and what makes data a "good idea", where a "good idea" is an abstraction that correlates with reality - people record observations and those recordings (of the past) are data. Other than in this conversation apparently, if you say you have some data, I know what you mean (some recorded observations). Interpretation of those observations is completely orthogonal.

Nothing about the idea of 'data' implies a lack of formatting/labeling/use of common language to convey the facts/observations, in fact it requires it. Data is not merely a signal and that is why we have two different ideas/words. '42' is not, itself, a fact (datum). What constitutes minimal sufficiency of 'data' is a useful and interesting question. E.g. should data always incorporate time, what are the tradeoffs of labeling being in- or out-of-band, per datom or dataset, how to handle provenance etc. That has nothing to do with data as an idea and everything to do with representing data well.

But equating any such labeling with more general interpretation is a mistake. For instance, putting facts behind a dynamic interpreter (one that could answer the same question differently at different times, mix facts with opinions/derivations or have effects) certainly exceeds (and breaks) the idea of data. Which is precisely why we need the idea of data, so we can differentiate and talk about when that is and is not happening - am I dealing with facts, an immutable observation of the past ("the king is dead") or just a temporary (derived) opinions ("there may be a revolt"). Consider the difference between a calculation involving (several times) a fact (date-of-birth) vs a live-updated derivation (age). The latter can produce results that don't add up. 'date-of-birth' is data and 'age' (unless temporally-qualified, 'as-of') is not.

When interacting with an ambassador one may or may not get the facts, and may get different answers at different times. And one must always fear that some question you ask will start a war. Science couldn't have happened if consuming and reasoning about data had that irreproducibility and risk.

'Data' is not a universal idea, i.e. a single primordial idea that encompasses all things. But the idea that dynamic objects/ambassadors (whatever their other utility) can substitute for facts (data) is a bad idea (does not correspond to reality). Facts are things that have happened, and things that have happened have happened (are not opinions), cannot change and cannot introduce new effects. Data/facts are not in any way dynamic (they are accreting, that's all). Sometimes we want the facts, and other times we want someone to discuss them with. That's why there is more than one good idea.

Data is as bad an idea as numbers, facts and record keeping. These are all great ideas that can be realized more or less well. I would certainly agree that data (the maintenance of facts) has been bungled badly in programming thus far, and lay no small part of the blame on object- and place-oriented programming.

richhickey

10 ปีที่แล้วdiscuss

It contravenes the common and historical use of the word 'data' to imply undifferentiated bits/scribbles. It means facts/observations/measurements/information and you must at least grant it sufficient formatting and metadata to satisfy that definition. The fact that most data requires some human involvement for interpretation (e.g. pointing the right program at the right data) in no way negates its utility (we've learned a lot about the universe by recording data and analyzing it over the centuries), even though it may be insufficient for some bootstrapping system you envision.

richhickey

10 ปีที่แล้วdiscuss

Data, and the entirety of human understanding and knowledge derived from recording, measurement and analysis of data, predates computing, so I don't see the relevance of these recent, programming-centric notions in a discussion of its value.

richhickey

10 ปีที่แล้วdiscuss

Without the 'idea' of data we couldn't even have a conversation about what interpreters interpret. How could it be a "really bad" idea? Data needn't be accompanied by an interpreter. I'm not saying that interpreters are unimportant/uninteresting, but they are separate. Nor have I said or implied that data is inherently meaningful.

Take a stream of data from a seismometer. The seismometer might just record a stream of numbers. It might put them on a disk. Completely separate from that, some person or process, given the numbers and the provenance alone (these numbers are from a seismometer), might declare "there is an earthquake coming". But no object sent an "earthquake coming" "message". The seismometer doesn't "know" an earthquake is coming (nor does the earth, the source of the 'messages' it records), so it can't send a "message" incorporating that "meaning". There is no negotiation or direct connection between the source and the interpretation.

We will soon be drowning in a world of IoT sensors sending context-or-provenance-tagged but otherwise semantic-free data (necessarily, due to constraints, without accompanying interpreters) whose implications will only be determined by downstream statistical processing, aggregation etc, not semantic-rich messaging.

If you meant to convey "data alone makes for weak messages/ambassadors", well ok. But richer messages will just bottom out at more data (context metadata, semantic tagging, all more data) Ditto, as someone else said, any accompanying interpreter (e.g. bytecode? - more data needing interpretation/execution). Data remains a perfectly useful and more fundamental idea than "message". In any case, I thought we were talking about data, not objects. I don't think there is a conflict between these ideas.

richhickey

10 ปีที่แล้วdiscuss

Data without an interpreter is certainly subject to (multiple) interpretation :) For instance, the implications of your sentence weren't clear to me, in spite of it being in English (evidently, not indicated otherwise). Some metadata indicated to me that you said it (should I trust that?), and when. But these seem to be questions of quality of representation/conveyance/provenance (agreed, important) rather than critiques of data as an idea. Yes, there is a notion of sufficiency ('42' isn't data).

Data is an old and fundamental idea. Machine interpretation of un- or under-structured data is fueling a ton of utility for society. None of the inputs to our sensory systems are accompanied by explanations of their meaning. Data - something given, seems the raw material of pretty much everything else interesting, and interpreters are secondary, and perhaps essentially, varied.

richhickey

10 ปีที่แล้วdiscuss

Data like that sentence? Or all of the other sentences in this chat? I find 'data' hard to consider a bad idea in and of itself, i.e. if data == information, records of things known/uttered at a point in time. Could you talk more about data being a bad idea?

richhickey

12 ปีที่แล้วdiscuss

It seems counterintuitive, but composing transducers yields a reducing function that runs the transformation steps left->right. What you are composing is the reducing function transformations (right to left, ordinary composition), but their result, having been built inside-out, runs the steps outside-in. So (comp tx1 tx2...) runs in same order as (->> xs tx1 tx2...)

richhickey

12 ปีที่แล้วdiscuss

While not mentioned in the blog post, the transducers implementation supports both early termination and result completion/flushing/cleanup.

richhickey

12 ปีที่แล้วdiscuss

I'm not yet seeing that, given:

Signal a :: Time -> a SF a b :: Signal a -> Signal b thus (Time -> a) -> (Time -> b)

not exactly:

(x->a->x) -> (x->b->x)

Can you point me to a paper that makes the connection clear?

richhickey

12 ปีที่แล้วdiscuss

Because mapcat's signature was not amenable to the additional arity, there's now also flatmap (note you can write the lazy collection version of any transducer fn using sequence as below):

    (defn flatmap
      "maps f over coll and concatenates the results.  Thus function f
      should return a collection.  Returns a transducer when no collection
      is provided."
      ([f]
       (fn [f1]
         (fn
           ([] (f1))
           ([result] (f1 result))
           ([result input]
              (reduce (preserving-reduced f1) result (f input))))))
    
      ([f coll] (sequence (flatmap f) coll)))

richhickey

12 ปีที่แล้วdiscuss

Yes, it's not just the 'reduce' function. You can think of many kinds of jobs in terms of seeded left reductions. Here's an example of some of the functions that can apply a transducer to their internal 'step' operation:

https://gist.github.com/richhickey/b5aefa622180681e1c81

Note how one transducer stack is created and can be reused in many different contexts.

richhickey

12 ปีที่แล้วdiscuss

I'm not seeing anything that looks like a reducing function transformer there. That all looks like variants of ordinary function composition, currying and partial application. Is there someplace that shows 'operator forms' acting as functions with this signature: (x->a->x)->(x->a->x)?

richhickey

12 ปีที่แล้วdiscuss

Kind of. The idea is to get out of the context of the 'whole job' (the ->r->r bit above) and focus on transformations of the step function (a->r->r) -> (b->r->r) {using your arg order above}. Not talking about the whole job (i.e. the source and result) makes for much more highly reusable components, especially when the jobs don't produce concrete results but, e.g., run indefinitely, like channel transformations.

richhickey

12 ปีที่แล้วdiscuss

Precisely the opposite - these functions compose because they are representation-free.

richhickey

12 ปีที่แล้วdiscuss

These sigs are for the arities below only.

map f: (a->b)->(x->b->x)->(x->a->x)

filter pred: (a->bool)->(x->a->x)->(x->a->x)

flatmap f: (a->[b])->(x->b->x)->(x->a->x)

etc.

comments