UW3 release candidate #32
Open
+297,140
−13,288
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Wrapper function for the access manager to improve usability
Minor deetails.
Attempting to fix up the evaluation to be internally consistent, adding the wrappers for the access manager (and calling it `array` for now). Fix up tests to account for changes to evaluation array sizes on return (mostly squeezes) Fix a couple of other evaluation bugs (like the need to add a zero vector to some nearly-constant matrices). Repairing the quickstart guide here and there.
Fix for the first comment by @jcgraciosa. Also needed to update the JIT code to understand the MATRIX variable type and removing the other types that are (currently) not used. These cause compilation errors if they are lit up by mistake (even if that variable is not in any expression). Solvers: Navier Stokes / Advection-Diffusion - fix the projection terms and how they pack up their data.
…orldcode/underworld3 into global_evaluation_rewrite
Adding global evaluation routines to compute values anywhere in the distributed domain (also with extrapolation using rbf)
Fixing broken test (array shape)
Not setting initial rank - this has problems when the particle is located on the correct rank because it automatically, always gets migrated to rank 0 and we start hunting for it again (but always in the wrong place).
Missed one !
Nodal Point Swarm also had the issue of uninitialised rank. This causes particles on rank > 1 to be migrated to rank 0 on the first call to migrate. They then get lost ... Potential fix @jcgraciosa but maybe not the only issue here.
Fixing lack of rank initialisation in swarm.populate. Also: some changes to the derivative expressions code: - .where relevant sym returns the inner expression derived by the diff_variable inner expression in symbolic form (which is closer to the intent of .sym) and does not return the evaluated form. - .doit function returns the evaluated form (so this is like the sympy deferred derivative).
…tions And a couple of other fixes
This code is *probably* no longer needed
And leaving NodalPointPICSwarm alone
Shortening some monster-sized files. Imports etc should not change
…t in a more user friendly form attached to the swarm
Follow ReadTheDocs recommended pattern using jobs hooks and asdf to install pixi, rather than custom curl commands. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
With pixi-based builds on ReadTheDocs, we have a real PETSc environment so mocking is no longer needed and was actually interfering with Cython module introspection. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Launcher repo can set ENV UW3_BRANCH to pull different branches. Defaults to uw3-release-candidate for backward compatibility. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Use --force-reinstall to ensure .so files are rebuilt after git pull - Show last 5 lines of build output for debugging - Update BINDER_CONTAINER_SETUP.md with launcher repo documentation
Sphinx was importing from source instead of installed package, causing circular import in function/__init__.py -> analytic. The package is already installed by 'pixi run build' before docs build. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add Limitations and Requirements section to BINDER_CONTAINER_SETUP.md - Document mybinder.org 30-second timeout constraint - Document when Cython extension changes require image rebuild - Revert start script to simple build (fix baked into image) - Update image rebuild procedure to use --no-cache flag Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Added API docs for: - function: Expressions, quantities, evaluation - swarm: Particle swarms and variables - scaling: Units, quantities, non-dimensionalisation - maths: Mathematical operations and integrals - materials: Multi-material systems - model: Model management and configuration - utilities: I/O and helper functions - visualisation: Plotting tools - adaptivity: Adaptive mesh refinement 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- container/Dockerfile.base.optimized: New Dockerfile that removes unnecessary files at build time (.git, headers, toolchain, docs_legacy) - Updated BINDER_CONTAINER_SETUP.md to document slim image variant and updated build/push procedures Size reduction achieved by removing: - .git directory (356MB) - include/ headers (390MB) - x86_64-conda-linux-gnu toolchain (267MB) - docs_legacy (229MB) - conda-meta, __pycache__, pip cache Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Instead of removing .git entirely (which required slow re-init), keep the shallow clone (depth 1, ~5MB). The start script can then do a fast `git pull --depth 1` to get latest Python code changes. This balances: - Small image size (shallow .git is ~5MB vs 356MB full) - Fast startup (git pull on shallow clone is quick) - Live updates (users get latest code, not frozen at build time) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Meshing now includes all 18 exported functions: - Cartesian: StructuredQuadBox, UnstructuredSimplexBox, BoxInternalBoundary - Annulus: Annulus, QuarterAnnulus, SegmentofAnnulus, AnnulusWithSpokes, AnnulusInternalBoundary, DiscInternalBoundaries - Spherical: SphericalShell, CubedSphere, SegmentofSphere, SphericalShellInternalBoundary - Geographic: RegionalSphericalBox, RegionalGeographicBox - Segmented: SegmentedSphericalShell, SegmentedSphericalSurface2D, SegmentedSphericalBall - Faults: FaultSurface, FaultCollection Function now includes global_evaluate and evaluate_gradient. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
New script scripts/docs_audit.py compares exported items vs documented items and reports gaps. Run with: pixi run -e dev docs-audit 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
mybinder.org struggles with layers > 1GB. Split the single large COPY into multiple smaller copies: - Pixi binary (~10MB) - Jupyter config (~1MB) - Runtime bin/share/ssl/etc (~100MB) - Runtime lib (~400MB) - Source code (~100MB) - Docs (~50MB) - Git (~5MB) Each layer is now under 500MB for reliable uploads. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Fixes: - Keep include/ and x86_64-conda-linux-gnu for runtime JIT compilation - Add Mesa software rendering env vars to fix VTK/EGL errors: - LIBGL_ALWAYS_SOFTWARE=1 - MESA_GL_VERSION_OVERRIDE=3.3 - GALLIUM_DRIVER=llvmpipe - VTK_DEFAULT_OPENGL_WINDOW=vtkOSOpenGLRenderWindow Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add libexec directory to Dockerfile for cc1 compiler component - Fix SyntaxWarnings in ddt.py (add raw string prefix for LaTeX math) - Create Dockerfile.deps for dependencies-only base image - Create Dockerfile.branch for branch-specific builds on top of deps - Update BINDER_CONTAINER_SETUP.md with two-layer architecture docs - Add troubleshooting for cc1 not found error The two-layer architecture allows: - Dependencies image to be cached (rarely changes) - Quick rebuilds for code changes - Multiple branches sharing the same deps image Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add jupyter-server-proxy to runtime dependencies - Configure trame server_proxy_enabled for remote environments - Set server_proxy_prefix to /proxy/ for binder compatibility - Detect JUPYTERHUB_SERVICE_PREFIX in addition to binder env vars Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Documentation: - Add high/medium priority items to API docs (Mesh, MeshVariable, UnderworldFunction, Surface classes, mesh adaptation, utilities) - Add visualisation module documentation with PyVista functions - Improve module-level descriptions and section organization Fix RST/Sphinx compatibility: - Convert Unicode math symbols to LaTeX ($\nabla$, $\theta$, etc.) - Replace pipe-delimited absolute values with \lvert...\rvert - Convert Markdown tables to RST-friendly list formats - Fix substitution reference errors in docstrings 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Let PyVista/trame auto-detect the server proxy prefix instead of hardcoding /proxy/ which caused 424 errors on mybinder.org. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Split the 2.7GB lib directory into 6 layers <800MB each for more reliable mybinder.org uploads. Previous 2.8GB single layer was causing upload failures. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add layer size constraints section (mybinder.org ~1GB limit) - Document lib directory splitting strategy for reliable uploads - Add trame/PyVista proxy configuration troubleshooting (no explicit prefix) - Update image sizes and add current SHA reference - Add GitHub Actions automation proposal for future CI/CD - Update environment variables for Mesa software rendering - Simplify start script documentation to match current implementation Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add nbgitpuller to base Docker image for cloning any content repo - Create binder_wizard.py script for generating launch URLs and badges - Update documentation with generic launcher architecture - Any repository can now use UW3 environment without .binder/ setup Usage: python scripts/binder_wizard.py # Interactive wizard python scripts/binder_wizard.py user/repo branch # Quick mode Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Direct binder launches should use the launcher repo which provides stable caching. The .binder here would bust the cache on every commit. Launch via: https://mybinder.org/v2/gh/underworldcode/uw3-binder-launcher/uw3-release-candidate Or use scripts/binder_wizard.py to generate badges for any content. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Remove scratch/test notebooks (Untitled.ipynb, Untitled1.ipynb, TestUnits.ipynb) - Remove draft notebooks (XX-Scaled_Thermal_Convection.ipynb, XX-Timestepping-with-units.ipynb) - Remove Jupytext .py pairings (2-Variables.py, 11-Multi-Material_SolCx.py, etc.) - Clean jupytext metadata from notebook files to prevent pairing recreation - Fix broken cross-references in Notebook_Index.ipynb - Add missing link to Notebook 15 (Rayleigh-Bénard Convection) 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Complete migration of user documentation to unified Sphinx build: - Convert 57 .qmd files to MyST-compatible .md format - Strip notebook outputs to avoid docutils rendering issues - Add Sphinx conf.py with myst-nb, sphinx-design, Furo theme - Create new landing page with sphinx-design grids/cards - Add sphinx-design dependency to pixi.toml - Remove Quarto config files (_quarto.yml, _variables.yml, theme.scss) - Delete presentations/slides directories (handled separately) Documentation structure: - beginner/ - User tutorials and guides (enabled) - advanced/ - Researcher guides (enabled) - api/ - API reference (enabled) - developer/ - Developer docs (excluded, needs cleanup) Build with: sphinx-build -b html docs docs/_build/html 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
When asked to document features, add to docs/developer/ with: - MyST Markdown format (Sphinx-compatible) - Standalone, concise explanations with code examples - Proper integration into toctree structure Also updated format reference from Quarto to Sphinx/MyST. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Fix Dockerfile path (container/Dockerfile.base.optimized) - Switch from DockerHub to GHCR - Add path triggers for Cython/dependency changes - Add workflow_dispatch for manual builds - Add cross-repo dispatch to update launcher Developed by Underworld Team with AI assistance (Claude)
- Revert docker-image.yml to original (DockerHub command-line builds) - Add binder-image.yml for mybinder.org image builds (GHCR) - Add container/README.md explaining binder Dockerfile architecture The two workflows serve different purposes: - docker-image.yml: Command-line Docker for local users (DockerHub) - binder-image.yml: Binder base image for mybinder.org (GHCR) Developed by Underworld Team with AI assistance (Claude)
- Add Containerfile for command-line Docker/Podman builds - Add launch-container.sh for rootless Podman launches - Update README.md to document both command-line and binder containers All container files now live in container/ rather than docs/developer/container/ Developed by Underworld Team with AI assistance (Claude)
- docker-image.yml → "DockerHub: Command-Line Container" - binder-image.yml → "GHCR: Binder Base Image" Developed by Underworld Team with AI assistance (Claude)
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
The full release of UW3 should be at 3.0 ... when we do this, I would like to introduce some new coding patterns that are more conducive to the having and AI-bot (AI-Bat ?) give advice.
So that means there is a lot to review in here.