It’s a POSIX thing, and it has nothing to do with the file extension. The OP is right: file extensions are optional (and, as shown here, often actively misleading), but if they’re present they should match the actual file type. Which, in this case, is Python code , not a shell script.
Oh, I totally agree that ‘renv’ probably solves 95% of problems. But those pesky 5%…
I think that most problems are ultimately caused by the fact that R packages cannot really declare versioned dependencies (most packages only declare `>=` dependency, even though they could also give upper bounds [1]; and that is woefully insufficient), and installing a package’s dependencies will (almost?) always install the latest versions, which may be incompatible with other packages. But at any rate ‘renv’ currently seems to ignore upper bounds: e.g. if I specify `Imports: dplyr (>= 0.8), dplyr (< 1.0)` it will blithely install v1.1.3.
The single one thing that causes most issues for us at work is a binary package compilation issue: the `configure` file for ‘httpuv’ clashes with our environment configuration, which is based on Gentoo Prefix and environment modules. Even though the `configure` file doesn’t hard-code any paths, it consistently finds the wrong paths for some system dependencies (including autotools). According to the system administrators of our compute cluster this is a bug in ‘httpuv’ (I don’t understand the details, and the configuration files look superficially correct to me, but I haven’t tried debugging them in detail, due to their complexity). But even if it were fixed, the issue would obviously persist for ‘renv’ projects requiring old versions.
(We are in the process of introducing a shared ‘renv’ package cache; once that’s done, the particular issue with ‘httpuv’ will be alleviated, since we can manually add precompiled versions of ‘httpuv’, built using our workaround, to that cache.)
Another issue is that ‘renv’ attempts to infer dependencies rather than having the user declare them explicitly (a la pyproject.toml dependencies), and this is inherently error-prone. I know this behaviour can be changed via `settings$snapshot.type("explicit")` but I think some of the issues we’re having are exacerbated by this default, since `renv::status()` doesn’t show which ones are direct and which are transitive dependencies.
Lastly, we’ve had to deactivate ‘renv’ sandboxing since our default library is rather beefy and resides on NFS, and initialising the sandbox makes loading ‘renv’ projects prohibitively slow — every R start takes well over a minute. Of course this is really a configuration issue: as far as I am concerned, the default R library should only include base and recommended packages. But it in my experience it is incredibly common for shared compute environments to push lots of packages into the default library. :-(
---
[1] R-exts: “A package or ‘R’ can appear more than once in the ‘Depends’ field, for example to give upper and lower bounds on acceptable versions.”
> But in all honesty for the pharmaceutical industry it’s mostly momentum that keeps R on top
I can’t agree with this: especially in PK/PD, R is only just now taking over from the previous (closed-source) systems. Momentum would keep R out, not in.
> I can’t remember the last time a library incompatibility led to a show stopper.
Oh, it’s very common unless you basically only use < 5 packages that are completely stable and no longer actively developed: packages break backwards compatibility all the time, in small and in big ways, and version pinning in R categorically does not work as well as in Python, despite all the issues with the latter. People joke about the complex packaging ecosystem in Python but at least there is such a thing. R has no equivalent. In Python, if you have a versioned lockfile, anybody can redeploy your code unless a system dependency broke. In R, even with an ‘renv’ lockfile, installing the correct packages version is a crapshoot, and will frequently fail. Don’t get me wrong, ‘renv’ has made things much better (and ‘rig’ and PPM also help in small but important ways). But it’s still dire. At work we are facing these issues every other week on some code base.
Indeed, that wouldn’t be a lot: the framing in the article is grossly misleading. The actual number of protesters this weekend was hundreds of thousands, and plausibly >1M (not “tens of thousands”), according to unanimous reporting in various German media and official sources (see e.g. https://www.tagesschau.de/inland/demos-gegen-rechts-bilanz-1...).
I’m not saying this doesn’t happen but I can’t remember ever having been asked to install a root certificate when joining an airport wifi. And I am confident that this has never happened when I’ve flown out from Gatwick.
Because we hadn’t heard about this case before. — At all. If the vague information in this article is actually corroborated there’ll be an outcry alright.
Why? Because all typical, existing applications written for POSIX systems use the POSIX API (usually indirectly) to interact with the filesystem and perform IO. Being able to use that vast ecosystem of existing applications seamlessly with object storage is such a common requirement that a good dozen different object storage-to-POSIX abstraction layers exist (many of them on top of FUSE).
Even if you build your scalable production environment from scratch with native object storage support (which is rare!), it’s eminently useful to be able to use coreutils to interact with said object storage for devops: nothing beats e.g. running grep on a list of log “files” in an S3 bucket in terms of convenience.
> … at Google, which i suspect is one of the companies you are referring to.
Just to clarify: no, I was referring to companies which are using products from the company behind cunoFS, and which share the actual code base of the functional interposition. My point is that while this technique is complex and brittle in general, this specific code-base is incredibly battle-tested and has proved itself even in fairly arcane configurations.
You’re definitely right about there being some amount of resistance, but functional interposition offers some compelling advantages over all alternative solutions in terms of ease of use and unparalleled performance.
> this is not a recipe for stability of your binaries
I see the point in theory, but it works incredibly well in practice. The specific technology has been used in production by major companies for years. It even carefully works around buggy software that makes incorrect assumptions for undefined behaviour.
(COI disclaimer: I used to work on this product; but I no longer have any stakes in it, financial or otherwise. I do still use the product, because it’s vastly superior to the alternatives.)
The article is about POSIX compliance (or lack thereof) of storage systems.
And the article does not imply that “POSIX … sucks”. On the contrary: the answer to the rhetoric question in the title is obviously “no”.
> It's almost always a better idea … to write your own thin specialized glue layer between your application code and operating system APIs.
Oh, definitely. But writing good abstractions that work equally well with POSIX-compliant filesystems and with object storage is basically impossible without massive trade-offs. That’s the entire point of the product that’s advertised here: it provides a link between object storage and POSIX-compliant file access that manages these trade-offs extremely well, and it allows users to use their existing glue code for POSIX without having to deal with object storage altogether.
(COI disclaimer: I used to work on this product; but I no longer have any stakes in it, financial or otherwise.)
If you save the offset (and you are, if you use a borrowed string) you can retrieve the original source code by re-reading the input file on the fly. This is expensive (due to IO, but especially also since you need to re-lex and recompute all offsets up to the point of the error), but it only needs to happen in case a parse error is generated, so the cost is likely negligible.
I’ve done this in a parser where I wanted to avoid storing line and column numbers instead of a single byte offset, but still wanted to provide the more user-friendly line and column numbers in the error message.
It’s also completely untrue. In reality the high cost is not because of greed but because of high manufacturing cost and the necessity for long hospital follow-up. Novartis is also not the only manufacturer providing CAR-T drugs at a high price. And lastly, they offer alternative payment programmes to make the therapy more affordable. https://en.wikipedia.org/wiki/CAR_T_cell#Economics
> I recall when the pipe operator was first being proposed the argument for it was that it'd enable workflows that felt more like Excel.
Where are you getting that from? To start with the pipe operator has been independently reinvented multiple times in R, and neither ‘magrittr’ nor ‘dplyr’ were the first to introduce the pipe operator into R. And (at least when I was exposed to it), the pipe operator had nothing whatsoever to do with Excel. Instead, it was an attempt to introduce the composability concepts from the UNIX shell and Haskell composition into R.
Yes, it switches to a different audio codec, that’s unavoidable due to bandwidth limitations. However, macOS insists on using a worse codec than necessary (SCO instead of SBC) and, at least with my MX4 headphones, this badly scrambles audio, at least intermittently. I’ve stopped using the headphone microphone because it was unbearable. I’ve tried playing with the bluetooth settings but nothing seems to fix this.
I’m a huge fan of the Sony high-end line, and am currently using the WH-1000XM4 myself. They’re excellent, but they do share quite a few of the annoyances listed in the OP article (in particular the delay when going from call to audio and vice versa, and the audio quality degradation; though, truth be told, I believe that’s a macOS issue), and they have their own annoyances. Most prominently, the utterly useless “second device pairing” audio message that plays for a full four seconds and drowns out whatever else you were listening to. I routinely need to ask people on a call to repeat what they said because of this. Yet Sony arrogantly states that this message is somehow “important” and therefore can’t be disabled.
It’s a weird quote anyway since there is commercial, proprietary software for DNA sequence analysis. Just a few examples of companies in this space are Sentieon, Edico (acquired by Illumina) and Parabricks (acquired by Nvidia). And Michael knows this (they’re sufficiently well known, and his own research laid some of the earliest foundations that Parabricks would ultimately build upon) so I’m assuming the quote was taken out of context or he was talking specifically about his own lab.
[ my public key: https://keybase.io/klmr; my proof: https://keybase.io/klmr/sigs/F9zackvFn6PE00C5T_IlAkRQPH_CVvp0aqSA-hysJ30 ]