Ir al contenido

Kernels

De Wikicima

En esta sección se detallan los pasos seguidos para instalar librerías y/o programario específico. Dichas librerías se tendrán que instalar para cada usuarie desde la sesión jupyter.

Kernels

Digital Earths - Global Hackathon

En la semana del 13 al 16 de Mayo aconteció el la Digital Earths - Global Hackathon en la cuál el CIMA/IFAECI participó con la organización de un nodo en CABA. Para esa actividad fue necesaria la instalación de un entorno de python el cuál se hizo accesible para todes les usuaries de Deimos. Estos son los pasos seguidos:

First download the Hachkathon's repository as root in /opt/onda/mamba/DE-global_hackathon (as root) 

# git clone https://github.com/digital-earths-global-hackathon/tools.git

Go to directory with installation and make a copy for deimos and edit the prefix 

# cd tools/python_envs
# cp install_python install_python_deimos
diff install_python install_python_orig install_python install_python_orig
4c4,5
< prefix=${prefix:-$HOME/python_envs}
---
> #prefix=${prefix:-$HOME/python_envs}
> prefix=/opt/conda/mamba/DE-global_hackathon
./install_python_deimos
```
We got the right installation
<PRE style="shell">
ls /opt/conda/mamba/DE-global_hackathon/
miniconda3  tools

Registering the new kernel jupyerhub-doc and multiple-kernels 

# /opt/tljh/user/bin/jupyter kernelspec list
Available kernels:
  python3            /opt/tljh/user/share/jupyter/kernels/python3
  bash               /usr/local/share/jupyter/kernels/bash
  ir                 /usr/local/share/jupyter/kernels/ir
  python_conda_su    /usr/local/share/jupyter/kernels/python_conda_su
# /opt/tljh/user/bin/jupyter-kernelspec remove python_conda_su
Kernel specs to remove:
  python_conda_su     	/usr/local/share/jupyter/kernels/python_conda_su
Remove 1 kernel specs [y/N]: y
# /opt/tljh/user/bin/jupyter-kernelspec list
Available kernels:
  python3    /opt/tljh/user/share/jupyter/kernels/python3
  bash       /usr/local/share/jupyter/kernels/bash
  ir         /usr/local/share/jupyter/kernels/ir
# /opt/conda/mamba/DE-global_hackathon/miniconda3/bin/mamba install ipykernel
# /opt/conda/mamba/DE-global_hackathon/miniconda3/envs/easy/bin/python3 -m ipykernel install
Installed kernelspec python3 in /usr/local/share/jupyter/kernels/python3
# /opt/tljh/user/bin/jupyter-kernelspec install /usr/local/share/jupyter/kernels/python3 --name=python_DigEarth-Hack
[InstallKernelSpec] Installed kernelspec python_digearth-hack in /usr/local/share/jupyter/kernels/python_digearth-hack
# /opt/tljh/user/bin/jupyter-kernelspec list
Available kernels:
  python3                 /opt/tljh/user/share/jupyter/kernels/python3
  bash                    /usr/local/share/jupyter/kernels/bash
  ir                      /usr/local/share/jupyter/kernels/ir
  python_digearth-hack    /usr/local/share/jupyter/kernels/python_digearth-hack

Editing kernel name to be recognizible in jupyterHUB api and set-up environment (see stackoverflow) 

# vim /usr/local/share/jupyter/kernels/python_digearth-hack/kernel.json
"display_name": "Python 3 (D.E. Hachkathon)",
"env": {
  "MAMBA_PREFIX":"/opt/conda/mamba/DE-global_hackathon/miniconda3/envs/easy",
  "MAMBA_DEFAULT_ENV":"easy",
  "PATH":"/opt/conda/mamba/DE-global_hackathon/miniconda3/envs/easy/bin:/opt/conda/mamba/DE-global_hackathon/miniconda3/condabin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
 },

Adding new packages/libraries to an existing environment

To add new libraries, using mamba (preferred way) 

# /opt/conda/mamba/DE-global_hackathon/miniconda3/bin/mamba install -n easy pyflextrkr

Via pip 

# cd /opt/conda/mamba/DE-global_hackathon/miniconda3/envs/easy
# bin/pip3 install healpy

ATRACKCS

El ATRACKCS Esta herramienta está diseñada para realizar seguimientos de sistemas atmosféricos. Estos son los pasos seguidos para su instalación en Deimos

Creando el enclave del paquete 

# cd /opt/conda/mamba/
# mkdir ATRACKCS
# cd ATRACKCS

Agarrando el software y preparando la instalación via la creación de un nuevo entorno de python 

# git clone https://github.com/alramirezca/ATRACKCS
# cd ATRACKCS
# /opt/conda/mamba/DE-global_hackathon/miniconda3/bin/mamba env create -f env_py3.yml

Activación del entorno y terminar la instalación 

# eval "$(/opt/conda/mamba/DE-global_hackathon/miniconda3/bin/mamba shell hook --shell bash)"
(base) # mamba activate atrackcs_py3
(atrackcs_py3) # pip install -e .

Registrando el nuevo kernel para deimos: 

(atrackcs_py3) # pip3 install ipykernel
(atrackcs_py3) # mamba deactivate
(base) # mamba deactivate
# /opt/tljh/user/bin/jupyter kernelspec list
Available kernels:
  python3                 /opt/tljh/user/share/jupyter/kernels/python3
  bash                    /usr/local/share/jupyter/kernels/bash
  ir                      /usr/local/share/jupyter/kernels/ir
  python_digearth-hack    /usr/local/share/jupyter/kernels/python_digearth-hack
# /opt/conda/mamba/DE-global_hackathon/miniconda3/envs/atrackcs_py3/bin/python3 -m ipykernel install
# /opt/tljh/user/bin/jupyter-kernelspec install /usr/local/share/jupyter/kernels/python3 --name=python_atrackcs
# /opt/tljh/user/bin/jupyter-kernelspec list
Available kernels:
  python3                 /opt/tljh/user/share/jupyter/kernels/python3
  bash                    /usr/local/share/jupyter/kernels/bash
  ir                      /usr/local/share/jupyter/kernels/ir
  python_atrackcs         /usr/local/share/jupyter/kernels/python_atrackcs
  python_digearth-hack    /usr/local/share/jupyter/kernels/python_digearth-hack

Editando el nombre del kernel para identificarlo desde deimos: 

# vim /usr/local/share/jupyter/kernels/python_atrackcs/kernel.json 
 "display_name": "Python 3 (atrackcs)",
 "env": {
  "MAMBA_PREFIX":"/opt/conda/mamba/DE-global_hackathon/miniconda3/envs/atrackcs_py3",
  "MAMBA_DEFAULT_ENV":"atrackcs_py3",
  "PATH":"/opt/conda/mamba/DE-global_hackathon/miniconda3/envs/atrackcs_py3/bin:/opt/conda/mamba/DE-global_hackathon/miniconda3/condabin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
 },

ESMValTool

Herramientas para el análisis de datos CMIP. ESMValTool es una herramienta en python, abierta y desarrollada por la comunidad.

Los pasos para su instalación se detallan acá /ESMValToolInst.

El sistema papa-deimos, se pudo constituir en parte, gracias a fondos del INSU - LEFE

ANDEX

Creación de un entorno para python3 desde cero.

Para eso vamos a utilizar el mamba que se instaló para la 'D.E. global Hackathon'

Localización de la creación del nuevo entorno de python 

# cd /opt/conda/mamba/
# mkdir ANDEX
# cd ANDEX

Creación del entorno de python llamado py3_andex a partir del archivo creado py3_andex_pkgs.yaml: 

# /opt/conda/mamba/DE-global_hackathon/miniconda3/bin/mamba create -n py3_andex -f py3_andex_pkgs.yaml

El archivo <CODDE>py3_andex_pkgs.yaml contiene: 

# cat py3_andex_pkgs.yaml
name: py3_andex
channels:
  - conda-forge
dependencies:
  - wrf-python
  - xesmf
  - netcdf4
  - dask
  - cartopy

Activación e instalación del paquete de kernels de jupyerHUB 

# eval "$(/opt/conda/mamba/DE-global_hackathon/miniconda3/bin/mamba shell hook --shell bash)"
# mamba activate atrackcs_py3
(py3_andex) # pip3 install ipykernel

Registro del nuevo kernel para jupyerHUB. Eso desde fuera del entorno, por eso lo desactivamos 

(py3_andex) # mamba deactivate
# /opt/tljh/user/bin/jupyter kernelspec list
Available kernels:
  python3                 /opt/tljh/user/share/jupyter/kernels/python3
  bash                    /usr/local/share/jupyter/kernels/bash
  ir                      /usr/local/share/jupyter/kernels/ir
  python_atrackcs         /usr/local/share/jupyter/kernels/python_atrackcs
  python_digearth-hack    /usr/local/share/jupyter/kernels/python_digearth-hack
  python_esmvaltools      /usr/local/share/jupyter/kernels/python_esmvaltools
# /opt/conda/mamba/DE-global_hackathon/miniconda3/envs/py3_andex/bin/python3 -m ipykernel install
# /opt/tljh/user/bin/jupyter-kernelspec install /usr/local/share/jupyter/kernels/python3 --name=py3_andex
# /opt/tljh/user/bin/jupyter-kernelspec list
Available kernels:
  python3                 /opt/tljh/user/share/jupyter/kernels/python3
  bash                    /usr/local/share/jupyter/kernels/bash
  ir                      /usr/local/share/jupyter/kernels/ir
  py3_andex               /usr/local/share/jupyter/kernels/py3_andex
  python_atrackcs         /usr/local/share/jupyter/kernels/python_atrackcs
  python_digearth-hack    /usr/local/share/jupyter/kernels/python_digearth-hack
# vim /usr/local/share/jupyter/kernels/py3_andex/kernel.json
{
 "argv": [
  "/opt/conda/mamba/DE-global_hackathon/miniconda3/envs/py3_andex/bin/python3",
  "-Xfrozen_modules=off",
  "-m",
  "ipykernel_launcher",
  "-f",
  "{connection_file}"
 ],
 "display_name": "Python 3 (ANDEX)",
 "env": {
  "MAMBA_PREFIX":"/opt/conda/mamba/DE-global_hackathon/miniconda3/envs/py3_andex",
  "MAMBA_DEFAULT_ENV":"py3_andex",
  "PATH":"/opt/conda/mamba/DE-global_hackathon/miniconda3/envs/py3_andex/bin:/opt/conda/mamba/DE-global_hackathon/miniconda3/condabin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
 },
 "language": "python",
 "metadata": {
  "debugger": true,
  "supported_encryption": "curve"
 },
 "kernel_protocol_version": "5.5"
}

Listado de paquetes en el entorno: 

(py3_andex) # pip3 freeze --local
asttokens==3.0.2
Cartopy==0.25.0
certifi==2026.6.17
cf_xarray @ file:///home/conda/feedstock_root/build_artifacts/cf_xarray_1781358794801/work
cftime==1.6.5
click==8.4.2
cloudpickle==3.1.2
comm==0.2.3
contourpy==1.3.3
cycler==0.12.1
dask==2026.7.1
debugpy==1.8.21
decorator==5.3.1
esmpy @ file:///home/conda/feedstock_root/build_artifacts/bld/rattler-build_esmpy_1767783521/work/src/addon/esmpy
executing==2.2.1
fonttools==4.63.0
fsspec==2026.6.0
ipykernel==7.3.0
ipython==9.15.0
ipython_pygments_lexers==1.1.1
jedi==0.20.0
jupyter_client==8.9.1
jupyter_core==5.9.1
kiwisolver==1.5.0
llvmlite==0.48.0
locket==1.0.0
matplotlib==3.11.0
matplotlib-inline==0.2.2
nest-asyncio2==1.7.2
netCDF4==1.7.4
numba @ file:///home/conda/feedstock_root/build_artifacts/numba_1784209014097/work
numpy @ file:///home/conda/feedstock_root/build_artifacts/bld/rattler-build_numpy_1779169198/work/dist/numpy-2.4.6-cp314-cp314-linux_x86_64.whl#sha256=67bfc5dc678227ea6ad6cc75838ae920b11bc7da7649fc19c624fda3aaec691a
packaging @ file:///home/conda/feedstock_root/build_artifacts/bld/rattler-build_packaging_1777103621/work
pandas @ file:///home/conda/feedstock_root/build_artifacts/bld/rattler-build_pandas_1778602611/work
parso==0.8.7
partd==1.4.2
pexpect==4.9.0
pillow==12.3.0
platformdirs==4.10.0
prompt_toolkit==3.0.52
psutil==7.2.2
ptyprocess==0.7.0
pure_eval==0.2.3
Pygments==2.20.0
pyparsing==3.3.2
pyproj==3.7.2
pyshp==3.1.4
python-dateutil @ file:///home/conda/feedstock_root/build_artifacts/bld/rattler-build_python-dateutil_1751104122/work
PyYAML==6.0.3
pyzmq==27.1.0
setuptools==83.0.0
shapely @ file:///home/conda/feedstock_root/build_artifacts/shapely_1762523525223/work
six @ file:///home/conda/feedstock_root/build_artifacts/bld/rattler-build_six_1753199211/work
sparse @ file:///home/conda/feedstock_root/build_artifacts/bld/rattler-build_sparse_1782985176/work
stack-data==0.6.3
toolz==1.1.0
tornado==6.5.7
traitlets==5.15.1
typing_extensions==4.16.0
wcwidth==0.8.2
wrapt @ file:///home/conda/feedstock_root/build_artifacts/wrapt_1782133551393/work
wrf-python @ file:///home/conda/feedstock_root/build_artifacts/wrf-python_1771438235011/work
xarray @ file:///home/conda/feedstock_root/build_artifacts/bld/rattler-build_xarray_1783633473/work
xesmf @ file:///home/conda/feedstock_root/build_artifacts/xesmf_1764332232756/work

>>> import os

os.environ["OMP_NUM_THREADS"] = "2" os.environ["MKL_NUM_THREADS"] = "2" os.environ["OPENBLAS_NUM_THREADS"] = "2" os.environ["VECLIB_MAXIMUM_THREADS"] = "2" os.environ["NUMEXPR_NUM_THREADS"] = "2"

import dask import numpy

  1. Set global constraints

dask.config.set(scheduler='threads', num_workers=2) >>>

Librerías

PyNCplot3

Librerías genéricas de Lluís Fita Borrell, CIMA PyNCplot

Des de una sesión bash del jupyterHUB: 

git clone -b numpy20 https://git.cima.fcen.uba.ar/lluis.fita/pyncplot.git PyNCplot3
cd PyNCplot3
ln -s Makefile.deimos-jupyterHub ./Makefile
make all >& run_make.log
ls *so
module_ForDef.cpython-39-x86_64-linux-gnu.so
module_ForDiag.cpython-39-x86_64-linux-gnu.so
module_ForDistriCorrect.cpython-39-x86_64-linux-gnu.so
module_ForGen.cpython-39-x86_64-linux-gnu.so
module_ForInt.cpython-39-x86_64-linux-gnu.so
module_ForSci.cpython-39-x86_64-linux-gnu.so

Después desde un directorio de trabajo del estudio concreto (e.j. EstudioXY12) almacenado dentro del directrio estudios del home del usuario de jupyteHUB. Desde la misma notebook de bash 

cd 
mkdir -p estudios/EstudioXY12
cd estudios/EstudioXY12
/home/jupyter-[usuario_JupyterHUB]/PyNCplot3/link_essentials_PWD.bash

Ahora ya se puede abrir un notebook the python en el directorio estudios/EstudioXY12 (aparecerá en el home del juyterHUB del usuario) y poder utilizar sin problemas las scripts de PyNCplot

Obtenido de «http://wiki.cima.fcen.uba.ar/index.php?title=Kernels&oldid=3797»