GeoData
#- export PYTHONPATH=.
#- export PYTHONPATH=.:/home/mrilee/git/Fumarole-dabi/fset/build
export PYTHONPATH=.:/home/mrilee/git/Fumarole-dabi/fset/build:/home/mrilee/opt/src/mrilee/git/NOGGIN-github/:/home/mrilee/opt/src/mrilee/git/CCL/
export PYTHONPATH=.:/home/mrilee/git/GeoData/geodata/:/home/mrilee/git/GeoData/
export PYTHONPATH=.:/home/mrilee/git/GeoData-dabi/geodata/:/home/mrilee/git/GeoData-dabi/
.:/home/mrilee/opt/src/mrilee/git/Fumarole/build-fset:/home/mrilee/opt/src/mrilee/git/STARE/build1/src/:/home/mrilee/opt/src/mrilee/git/NOGGIN-github/:/home/mrilee/opt/src/mrilee/git/CCL/:/home/mrilee/opt/src/mrilee/git/GeoData/
.:
home/mrilee/opt/src/mrilee/git/Fumarole/build-fset:
/home/mrilee/opt/src/mrilee/git/STARE/build1/src :
home/mrilee/opt/src/mrilee/git/NOGGIN-github :/home/mrilee/opt/src/mrilee/git/CCL/:
home/mrilee/opt/src/mrilee/git/GeoData
FERMAT_SDK_HOME=/home/mrilee/opt/src/mrilee/Fermat/FermatSDK20200603/
/home/mrilee/opt/src/mrilee/Fermat/FermatSDK20200603-1
export PYTHON_INCLUDE_DIRS=/home/mrilee/opt/anaconda3/envs/2022-work/include/python3.10:/home/mrilee/opt/anaconda3/envs/2022-work/lib/python3.10/site-packages/numpy/core/include/
home/mrilee/opt/anaconda3/envs/2022-work/include/python3.10
/home/mrilee/opt/anaconda3/envs/2022-work/lib/python3.10/site-packages/numpy/core/include
PYTHON_INCLUDE_DIRS=/home/mrilee/opt/anaconda3/include/python3.7m/:/home/mrilee/opt/anaconda3/lib/python3.7/site-packages/numpy/core/include/
PYTHON_INCLUDE_DIRS=
home/mrilee/opt/anaconda3/include/python3.7m :
home/mrilee/opt/anaconda3/lib/python3.7/site-packages/numpy/core/include
(insert (concat “\n” (shell-command-to-string “ls geodata/* | grep -v \~ “)))
geodata/geodata.py The main module of support routines. geodata/sketch1.py Load GOES IMG Band 5 data (NetCDF4) geodata/sketch10.py Load GOES SNDR Band 7 data geodata/sketch11.py Load MERRA-2 TPW data, scatterplot geodata/sketch12.py Cross comparison MERRA-2 and GOES IMG Band 5, plot correlation geodata/sketch13.py Cross comparison MERRA-2 and GOES IMG Band 3 geodata/sketch14.py Cross comparison MERRA-2 and GOES IMG Band 4 geodata/sketch2.py Synthetic data containing parameterized tracks geodata/sketch3.py Load and scatterplot MERRA-2 data geodata/sketch4.py Load and scatterplot AAOP/DMOP/MHOP alternative TPW data (MSPPS ORB) geodata/sketch5.py Load MERRA-2 times and create STARE temporal indices A big question here is are we working with intervals or points.
geodata/sketch6.py Work with STARE temporal ids from NetCDF4 datasets Do we want to work with intervals or points in time?
geodata/sketch7.py Look for intersections between MERRA and GOES files (reads files) geodata/sketch8.py Get STARE temporal id from filenames and compare (uses filenames) geodata/sketch9.py Construct a compliant h5 data file. geodata/sketch90.py Read from the h5 file and check the results of sketch9.py. geodata/sketch91.py Load GOES dataset from NetCDF4 files, load a few locations and check resolution. geodata/sketch92.py Load data from the h5 file and fit three components to the histogram. geodata/sketch93.py Load h5 data, like 92, but display the ratio b5[ge]/b5[lt] geodata/sketch94.py Load h5 data, like 92, but display image masked by m2 tpw at various thresholds geodata/sketch94a.py Load h5 data, like 92, but display image masked by b5 with various thresholds geodata/sketch95.py Load h5 data and try to compare b4-b5 against TPW Added root fitting to theoretical model that matches linear fit very well. Threshold of about -1850 found.
geodata/sketchA.py Check terminators at various resolutions, esp. 27. geodata/sketchB.py Go over sortedcontainers and iterators. geodata/sketchC.py DASK accelerates ps.from_latlon STARE calc. geodata/sketchD.py Stopwatch sketching, Don’t forget nx,ny <-> row,column, i.e. ny,nx… geodata/sketchE.py Try CCL. Used both cv2 calls and ccl_marker_stack. geodata/sketchE0.py Try to understand cv2.threshold geodata/sketchE1.py Try CCL and tracking with multiple h5 files. Fixed the color mapping. geodata/sketchE2.py Some clean up or refactoring of E1. CCL on b4-b5. geodata/sketchE3.py Refactor CCL/Tracking as a class. Adds various h5 outputs including label resolution. geodata/sketchF.py Try to make preprocessing into a callable function. geodata/sketchF0.py Add visualization to check previous. geodata/sketchF1.py Multi-file load and join, save to h5 files. geodata/sketchF2.py Load h5 files and display to check. geodata/sketchG.py Load MODIS data into a dataset geodata/sketchG0.py Check the MODIS data in h5 format, including GRING geodata/sketchG1.py Integrate GOES and MODIS data, show comparison viz. geodata/sketchG1-1.py Show a whole MODIS granule, superimposed on GOES in a lat-lon bbox. geodata/sketchG2.py Compare GOES and MODIS Granules by “clearing” bits to level, uses GRING, and distribute across nodes geodata/sketchH0.py Partition MODIS data, basic sketches for VDS geodata/sketchH1.py Shape and type check of VDS-style data stored in HDF geodata/sketchH2.py Visualize VDS-style partitioning of MODIS data geodata/sketchI0.py Intersection example geodata/sketchJ0.py Implement a new src_coord convention. geodata/sketchJ1.py Implement a new src_coord convention. Added to GeoData/geodata. geodata/sketchJ2.py Virtual dataset experiments geodata/sketchJ3.py Virtual dataset experiments geodata/sketchK0.py Load GOES & MODIS, lexsort, nadir vs. wing resolution - shows box on lower left and nadir geodata/sketchK0a.py Load GOES & MODIS, lexsort, nadir vs. wing resolution - plot 2 scans, allow zoom to show overlap geodata/sketchK1.py GOES+MODIS Template geodata/sketchK2.py GOES+MODIS - plot 2 scans, 2 subplots, full granule + box on left geodata/sketchL0.py Rework native array index encoding (src_coord, early version of sketchJ*.py?) geodata/sketchL1.py Rework native array index encoding (src_coord, early version of sketchJ*.py?) geodata/sketchM0.py Visualize root polyhedron Details on performing CCL on the GOES bands OpenCV.connectedComponents works on uint8 (only?) compare 1 f-dataset to per channel f-dataset