Maintenance and Development

Python Image Maintenance

Each version of python image is scheduled once a week for a trivy scan in Gitlab. It automatically checks, if libraries used contain know vulnerabilities, and provides list of affected libraries together with fixed versions. To fix the python version vX, take following steps:

1. Create a branch based on vX to update vX, usually "fix-trivy-scan-vX"

2. In "trivy-scan-vX", update the libraries, in the order: pyfx, datascience, neural

3. Make MR for branch into vX

In second step:
For Linux packages, those should be fixed automatically once image is rebuilt (It installs latest versions of python packages during build phase, so triggering pipeline should be enough)
For Python packages, versions of those are fixed in poetry.lock file, and they need to be updated there.
To do so, run command:

This requires poetry install successfully run previously. Without previous install, workaround might be used: Add that package into pyproject.toml and set its minimal version to one containing fix.
run "poetry lock --no-update", this will update lock file based on pyproject.toml, but not update other packages, effectively updating just ones mentioned, and other if necessary due to dependency.

Finally, add modified poetry.lock into fix-trivy-scan branch.

Image Development

Running python locally

Running python locally allows to test changes without re-running gitlab pipeline in each iteration.
To do so, in a folder with installed you can run poetry run python [script.py]
Alternatively you can add python installed by a poetry to your IDE as an interpreter, and use it from IDE.
To locate where poetry installed python, run: poetry run which python.
This will give you path to python, which you have to add into IDE as interpreter. Also, if you are using Python Console, and made changes to pyfx, you will need to restart the console for changes to take effect.

Using job context