Merge pull request #1 from gristlabs/ext

Inlining extension repo

README.md

@@ -1,29 +1,31 @@
-# JupyterLite Demo
+# JupyterLite Notebook Grist Custom Widget
 
-[![lite-badge](https://jupyterlite.rtfd.io/en/latest/_static/badge.svg)](https://jupyterlite.github.io/demo)
+See [USAGE.md](./USAGE.md) for instructions on how to use this widget in Grist. This README is for developers.
 
-JupyterLite deployed as a static site to GitHub Pages, for demo purposes.
+This repo is a custom deployment of JupyterLite generated from https://github.com/jupyterlite/demo.
 
-## ✨ Try it in your browser ✨
+## Development
 
-➡️ **https://jupyterlite.github.io/demo**
+1. Create and activate a virtual environment
+2. `pip install -r requirements.txt`
+3. In the `extension` folder, run `jlpm install`, then `jlpm build`, then `jlpm watch`. `jlpm` is a pinned version of `yarn` that is installed with JupyterLab, so you can use `yarn` or `npm` instead. `jlpm watch` will rebuild the extension when changes are made to the code under `extension/src`. For some reason it doesn't work without running `jlpm build` first at least once.
+4. In a new tab, back in the repo root, activate the virtual environment again, then run `./dev.sh`. This will start a local server at http://localhost:8000 which you can use as a custom widget URL.
+5. Make some changes to the code under `grist` or `extension/src`. Changing `.ts` files will rebuild the JS, but either way you still have to interrupt the server with Ctrl+C, rerun `.dev.sh`, and refresh the page to see the changes.
+6. If you're having trouble, try various permutations of these commands:
+   - `pip uninstall grist_jupyterlab_widget` (that's the Python package name of the `extension` folder)
+   - `pip install -e extension`
+   - `jupyter labextension develop ./extension` (part of `dev.sh`), maybe with the `--overwrite` flag.
+   - `jlpm build` and `jlpm watch` within the `extension` folder
 
-![github-pages](https://user-images.githubusercontent.com/591645/120649478-18258400-c47d-11eb-80e5-185e52ff2702.gif)
+## Deployment
 
-## Requirements
+Push changes to the `main` branch. The GitHub Action will build and publish to GitHub Packages.
 
-JupyterLite is being tested against modern web browsers:
+## Files
 
-- Firefox 90+
-- Chromium 89+
-
-## Deploy your JupyterLite website on GitHub Pages
-
-Check out the guide on the JupyterLite documentation: https://jupyterlite.readthedocs.io/en/latest/quickstart/deploy.html
-
-## Further Information and Updates
-
-For more info, keep an eye on the JupyterLite documentation:
-
-- How-to Guides: https://jupyterlite.readthedocs.io/en/latest/howto/index.html
-- Reference: https://jupyterlite.readthedocs.io/en/latest/reference/index.html
+- `extension/` contains the JupyterLab extension that connects the Grist and JupyterLab APIs. See the README there.
+- `grist/` contains most of the Python code that runs inside the JupyterLite Pyodide and that users can call.
+- `package.sh` packages the files under `grist` and puts them in `files/package.tar.gz`. JupyterLite picks up the contents of `files` when building, so the package can be downloaded from http://localhost:8000/files/package.tar.gz. `package.sh` is run by both `dev.sh` and the GitHub Action.
+- `extension/src/initKernelPy.ts` contains the 'bootstrapping' Python code that the extension runs in the kernel on startup. It downloads the package, extracts it, and imports it.
+- `dev.sh` cleans out old state, does some minimal building for development, and starts a local JupyterLite server.
+- `jupyter-lite.json` contains configuration for the JupyterLite deployment.

dev.sh

@@ -0,0 +1,8 @@
+#!/usr/bin/env bash
+
+set -eux
+
+rm -rf _output .jupyterlite.doit.db
+jupyter labextension develop ./extension
+./package.sh
+jupyter lite serve

extension/.copier-answers.yml

@@ -0,0 +1,19 @@
+# Changes here will be overwritten by Copier; NEVER EDIT MANUALLY
+_commit: v4.2.0
+_src_path: https://github.com/jupyterlab/extension-template
+author_email: [email protected]
+author_name: Alex Hall
+data_format: string
+file_extension: ''
+has_binder: false
+has_settings: false
+kind: frontend
+labextension_name: grist-widget
+mimetype: ''
+mimetype_name: ''
+project_short_description: Custom Grist widget for a JupyterLite notebook
+python_name: grist_jupyterlab_widget
+repository: ''
+test: false
+viewer_name: ''
+

extension/.gitignore

@@ -0,0 +1,121 @@
+*.bundle.*
+lib/
+node_modules/
+*.log
+.eslintcache
+.stylelintcache
+*.egg-info/
+.ipynb_checkpoints
+*.tsbuildinfo
+grist_jupyterlab_widget/labextension
+# Version file is handled by hatchling
+grist_jupyterlab_widget/_version.py
+
+# Created by https://www.gitignore.io/api/python
+# Edit at https://www.gitignore.io/?templates=python
+
+### Python ###
+# Byte-compiled / optimized / DLL files
+__pycache__/
+*.py[cod]
+*$py.class
+
+# C extensions
+*.so
+
+# Distribution / packaging
+.Python
+build/
+develop-eggs/
+dist/
+downloads/
+eggs/
+.eggs/
+lib/
+lib64/
+parts/
+sdist/
+var/
+wheels/
+pip-wheel-metadata/
+share/python-wheels/
+.installed.cfg
+*.egg
+MANIFEST
+
+# PyInstaller
+#  Usually these files are written by a python script from a template
+#  before PyInstaller builds the exe, so as to inject date/other infos into it.
+*.manifest
+*.spec
+
+# Installer logs
+pip-log.txt
+pip-delete-this-directory.txt
+
+# Unit test / coverage reports
+htmlcov/
+.tox/
+.nox/
+.coverage
+.coverage.*
+.cache
+nosetests.xml
+coverage/
+coverage.xml
+*.cover
+.hypothesis/
+.pytest_cache/
+
+# Translations
+*.mo
+*.pot
+
+# Scrapy stuff:
+.scrapy
+
+# Sphinx documentation
+docs/_build/
+
+# PyBuilder
+target/
+
+# pyenv
+.python-version
+
+# celery beat schedule file
+celerybeat-schedule
+
+# SageMath parsed files
+*.sage.py
+
+# Spyder project settings
+.spyderproject
+.spyproject
+
+# Rope project settings
+.ropeproject
+
+# Mr Developer
+.mr.developer.cfg
+.project
+.pydevproject
+
+# mkdocs documentation
+/site
+
+# mypy
+.mypy_cache/
+.dmypy.json
+dmypy.json
+
+# Pyre type checker
+.pyre/
+
+# End of https://www.gitignore.io/api/python
+
+# OSX files
+.DS_Store
+
+# Yarn cache
+.yarn/

extension/.prettierignore

@@ -0,0 +1,6 @@
+node_modules
+**/node_modules
+**/lib
+**/package.json
+!/package.json
+grist_jupyterlab_widget

extension/.yarnrc.yml

@@ -0,0 +1 @@
+nodeLinker: node-modules

extension/LICENSE

@@ -0,0 +1,29 @@
+BSD 3-Clause License
+
+Copyright (c) 2023, Alex Hall
+All rights reserved.
+
+Redistribution and use in source and binary forms, with or without
+modification, are permitted provided that the following conditions are met:
+
+1. Redistributions of source code must retain the above copyright notice, this
+   list of conditions and the following disclaimer.
+
+2. Redistributions in binary form must reproduce the above copyright notice,
+   this list of conditions and the following disclaimer in the documentation
+   and/or other materials provided with the distribution.
+
+3. Neither the name of the copyright holder nor the names of its
+   contributors may be used to endorse or promote products derived from
+   this software without specific prior written permission.
+
+THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
+AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
+DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE
+FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
+SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
+CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
+OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

extension/README.md

@@ -0,0 +1,63 @@
+# grist_jupyterlab_widget
+
+This is a JupyterLab extension that connects the Grist and JupyterLab APIs. It's tightly coupled with the JupyterLite deployment in this repo (the parent folder) and doesn't work on its own.
+
+This folder was originally [its own repo](https://github.com/gristlabs/jupyterlab-widget-extension) generated using `copier` following the [extension tutorial](https://jupyterlab.readthedocs.io/en/stable/extension/extension_tutorial.html). This is the source of a lot of boilerplate configuration that probably isn't *all* needed but also probably shouldn't be messed with. Usually this extension would be published on PyPI (and maybe NPM) under the package name `grist_jupyterlab_widget`, but now the parent folder just installs it from the local filesystem.
+
+## Code overview
+
+Most of the logic is in `src/index.ts`.
+
+1. The entrypoint is the exported `plugin: JupyterFrontEndPlugin` which JupyterLab picks up as an extension, running `activate(app: JupyterFrontEnd)` on startup.
+2. The extension adds a `<script>` tag for `grist-plugin-api.js` to the page and uses the grist API once it loads.
+3. `grist.getOption/setOption` and `app.serviceManager.contents` are used to save/load a notebook file in the widget options. All changes to the notebook are saved immediately.
+4. JupyterLite automatically starts a Pyodide (Python) kernel for the notebook. Once it's ready, we execute the Python code in `src/initKernelPy.ts` which bootstraps the rest of the Python code - see the parent README.
+5. The Pyodide kernel runs in a separate web worker. To give the user's Python code access to the `grist` API object in the main browser thread, the [`Comlink`](https://github.com/GoogleChromeLabs/comlink) library is used to `expose` the `grist` object to the worker. This requires the `Worker` object which the JupyterLab API doesn't provide, so the `Worker` constructor is monkeypatched to intercept its creation.
+
+## Development install
+
+Below are some of the original instructions included with this repo which may be helpful.
+
+Note: You will need NodeJS to build the extension package.
+
+The `jlpm` command is JupyterLab's pinned version of
+[yarn](https://yarnpkg.com/) that is installed with JupyterLab. You may use
+`yarn` or `npm` in lieu of `jlpm` below.
+
+```bash
+# Clone the repo to your local environment
+# Change directory to the grist_jupyterlab_widget directory
+# Install package in development mode
+pip install -e "."
+# Link your development version of the extension with JupyterLab
+jupyter labextension develop . --overwrite
+# Rebuild extension Typescript source after making changes
+jlpm build
+```
+
+You can watch the source directory and run JupyterLab at the same time in different terminals to watch for changes in the extension's source and automatically rebuild the extension.
+
+```bash
+# Watch the source directory in one terminal, automatically rebuilding when needed
+jlpm watch
+# Run JupyterLab in another terminal
+jupyter lab
+```
+
+With the watch command running, every saved change will immediately be built locally and available in your running JupyterLab. Refresh JupyterLab to load the change in your browser (you may need to wait several seconds for the extension to be rebuilt).
+
+By default, the `jlpm build` command generates the source maps for this extension to make it easier to debug using the browser dev tools. To also generate source maps for the JupyterLab core extensions, you can run the following command:
+
+```bash
+jupyter lab build --minimize=False
+```
+
+### Development uninstall
+
+```bash
+pip uninstall grist_jupyterlab_widget
+```
+
+In development mode, you will also need to remove the symlink created by `jupyter labextension develop`
+command. To find its location, you can run `jupyter labextension list` to figure out where the `labextensions`
+folder is located. Then you can remove the symlink named `grist-widget` within that folder.

extension/grist_jupyterlab_widget/__init__.py

@@ -0,0 +1,16 @@
+try:
+    from ._version import __version__
+except ImportError:
+    # Fallback when using the package in dev mode without installing
+    # in editable mode with pip. It is highly recommended to install
+    # the package from a stable release or in editable mode: https://pip.pypa.io/en/stable/topics/local-project-installs/#editable-installs
+    import warnings
+    warnings.warn("Importing 'grist_jupyterlab_widget' outside a proper installation.")
+    __version__ = "dev"
+
+
+def _jupyter_labextension_paths():
+    return [{
+        "src": "labextension",
+        "dest": "grist-widget"
+    }]

extension/install.json

@@ -0,0 +1,5 @@
+{
+  "packageManager": "python",
+  "packageName": "grist_jupyterlab_widget",
+  "uninstallInstructions": "Use your Python package manager (pip, conda, etc.) to uninstall the package grist_jupyterlab_widget"
+}