gvalkov · HackerTrans

gvalkov

no profile record

Submissions

Jenkins-autojobs: create jenkins jobs from git/svn/hg branches

gvalkov.github.com

4 pointsby gvalkov14 yıl önce1 comments

comments

2 ay öncediscuss

[dead]

2 ay öncediscuss

The squeeze is real even at the SME level. We recently wanted to add another TB of memory to several servers (we do EDA chip design, which eats a lot of memory). Quotes came back to about €200k for 48 x 96GB DDR5-5600 RDIMMs. Mind you, this is for refurbished memory with 1 year warranty. I'm still figuring out if this is FU-pricing or just how things are going to be from now on.

Spec-ing and buying servers has become quite the pain in the past year, at least at the relatively-small scale we operate at. It's "dynamic pricing" with most quotes being valid for 24 hours :(

4 ay öncediscuss

In zsh you can bind "push-line-or-edit". In bash and all readline programs, you can approximate it with C-u followed by C-y (i.e. cut and paste). My history is still full of '#' and ':' (csh trauma) prefixed command-lines like you described though ...

11 ay öncediscuss

I'm using that case with with a 200mm fan front-panel mod[1]. Despite its size, it gets pretty cramped. It's just hard to keep it organized without a dedicated SATA backplane.

[1]: https://www.printables.com/model/866109-200mm-fan-front-for-...

geçen yıldiscuss

In Python 3.8 - https://peps.python.org/pep-0572/

geçen yıldiscuss

This is nitpicking, but this is a good usecase for the := operator:

  if not (API_KEY := os.getenv("API_KEY")):
      ...

For internal tools I just let os.environ["API_KEY"] raise a KeyError. It's descriptive enough.

geçen yıldiscuss

We're still serving a cgi-bin directory at work for the occasional quick and dirty internal web app. The ergonomics are great as long as you keep it simple. The fact that it's cgi doesn't mean you have to print http/1.0 to stdout manually. For example, in python the builtin wsgiref.handlers.CGIHandler lets you run any wsgi app as a cgi script:

  import wsgiref.handlers, flask
  app = flask.Flask(__name__)
  wsgiref.handlers.CGIHandler().run(app)

The way we run the scripts is with uwsgi and its cgi plugin[1]. I find it simpler and more flexible than running apache or lighttpd just for mod_cgi. Since uwsgi runs as a systemd unit, we also have all of systemd's hardening and sandboxing capabilities at our disposal. Something very convenient in uwsgi's cgi handling that's missing from mod_cgi, is the ability to set the interpreter for a given file type:

  cgi = /cgi-bin=/webapps/cgi-bin/src
  cgi-allowed-ext = .py
  cgi-helper = .py=/webapps/cgi-bin/venv/bin/python3  # all dependencies go here

Time to first byte is 250-350ms, which is acceptable for our use case.

[1]: https://uwsgi-docs.readthedocs.io/en/latest/CGI.html

geçen yıldiscuss

I recently built something[1] similar, though with far less effort and sophistication than the author. The goal was to have a plug-and-play audiobook player for an elderly family member with impaired vision. In retrospect, it would have been better to adapt an old phone or tablet with a macropad rather than build this on top of an espmuse speaker[2].

I keep thinking that a cassette player would be the ideal interface for something like this. The controls are as obvious and as tactile as it gets and the whole analog-mechanical experience is familiar to folks from that generation. If only tapes could hold more than two hours of audio ...

[1]: https://www.printables.com/model/1269288-audiobook-player

[2]: https://raspiaudio.com/product/esp-muse-luxe/

4 yıl öncediscuss

Python 3.11 also claims [1] to be 10-60% faster than 3.10, which is the version that the benchmarks game is using at this time. The difference in used RSS is also quite interesting in this comparison.

  [1]: https://docs.python.org/3/whatsnew/3.11.html#whatsnew311-faster-cpython

4 yıl öncediscuss

A hidden gem of pyenv is its 'python-build' plugin, which just lets you build and install any Python version in the least number of steps:

  git clone https://github.com/pyenv/pyenv.git
  cd pyenv/plugins/python-build/bin
  ./python-build --definitions
  ./python-build 3.10.8 /opt/python/3.10.8
  PYTHON_CONFIGURE_OPTS="--enable-shared" ./python-build 3.10.8 /opt/python/3.10.8

4 yıl öncediscuss

One nice use of dd is to append an ssh key to .authorized_keys on a host that doesn't allow shell or sftp access (which is what ssh-copy-id needs):

  cat id_rsa.pub | ssh $host 'dd of=.ssh/authorized_keys oflag=append conv=notrunc'

5 yıl öncediscuss

It's great to see how far Podman (and its sister projects) have come. I think it's a reliable tool and I'm a happy user both personally and professionally.

We make heavy use of Podman in our infrastructure and it's mostly a pleasure. My current pet peeves are that:

1) Ansible's podman_container module is not as polished as docker_container. I regularly run into idempotency issues with it (so lots of needlessly restarted containers).

2) Gitlab's Docker executor doesn't support Podman and all our CI agents run on CentOS 8. I ended up writing a custom executor for it and it's working quite well though (we're probably not going back to the container executor even if it supported Podman, since the custom executor offers so much more flexibility).

3) GPU support is easier/more documented on Docker. For this reason, the GPU servers we have are all Ubuntu 20.04 + Docker since it's the more beaten path.

4) Podman-compose just needs more work. Luckily for us, it seems that Podman 3.x will support docker-compose natively [1].

As mentioned, our CI environment is very dependent on Podman. The first step of every Gitlab pipelines is to build the container image in which the rest of the jobs will run. I find that it's simpler to have a shell executor in a unprivileged, restricted environment (i.e. can only run `podman build`) than setting up dind just for building images. All jobs that follow are ran in rootless containers, for that nice added layer of security.

Wishing all the best to the Podman, Buildah and Skopeo teams.

[1]: https://www.redhat.com/sysadmin/podman-docker-compose

5 yıl öncediscuss

Quite a lot is possible with CNI [1]. For example, we use this setup to give real IPs to containers:

  # /etc/cni/net.d/testnet.conflist
  {
    "cniVersion": "0.4.0",
    "name": "testnet",
    "plugins": [
      {
        "type": "bridge",
        "bridge": "br0",  # main host interface is part of this bridge
        "ipam": {
          "type": "host-local",
          "subnet": "10.0.0.0/16",
          "gateway": "10.0.0.1",
          "routes": [{ "dst": "0.0.0.0/0"}]
        }
      }
    ]
  }

You can then start a container and operate on its network namespace for added flexibility:

  podman run -it --net testnet --ip 10.0.0.2 ...

  ns=$(basename $(podman inspect $id | jq -r '.[0] .NetworkSettings .SandboxKey'))
  ip netns exec $ns ip route add ...

[1]: https://github.com/containernetworking/cni

7 yıl öncediscuss

A device that waits for the SSID of my Olympus WiFi-enabled camera to show up and then sync all new photos:

https://github.com/gvalkov/olympus-photosync-server https://github.com/gvalkov/olympus-photosync

7 yıl öncediscuss

It's not possible in the way it's possible in SCons. You can basically use custom_target, but its usefulness is limited since there are no user-defined functions in Meson's subset of Python. CMake is a little better in this regard here, since it at least offers macros and functions.

7 yıl öncediscuss

Personally I see Meson as an attempt to do CMake right. A big differences between Meson and SCons is that SCons handles the execution of the build graph, while Meson delegates execution to Ninja. The nice thing about the former is that while the graph is being walked, new nodes can be added. The nice thing about the latter is that it's incredibly efficient.

7 yıl öncediscuss

I'm a build engineer and SCons is one of the gems in my tool belt. Where it truly shines is as a framework for building build systems. At its core, it's just a library for building and executing a DAG. What most SCons users work with is the standard-library of rules built on top of the core API, that makes it immediately usable as a "high-level" build system like Meson or Cmake. In my experience, it's unparalleled when you have to model an entirely custom build-flow in a clean way. I've used it to model build-flows for custom tool-chains that would have been a nightmare to reason about if they were written in GNU Make and outright impossible with a meta-build system.

The only other tools I've found to rival this flexibility are Gradle (see the Software Domain Modeling chapter of its documentation) and Shake (though having to write rules in Haskell makes it a hard pill to swallow).

8 yıl öncediscuss

  > Use it. Talk about it. Write about it.

I have a project that converts basic setup.py files to setup.cfg files [1].

Still happily using plain setuptools for library development and pip-tools for application development.

[1]: https://github.com/gvalkov/setuptools-py2cfg

8 yıl öncediscuss

A problem with color is that people end up optimizing the aesthetics for their own terminal setup. There is a wild number of different color schemes out there and it's really hard to make something that looks good on all of them. In my experience the only safe choice is bold text (i.e. \033[1m) since it stands out in all cases.

8 yıl öncediscuss

Several people have tried to tackle this in Python. My attempt was python-oneliner[1]. There's also a list of similar projects in the readme.

[1] https://github.com/gvalkov/python-oneliner