- auto write code into
sketch.js(incrementally) - look at errors and fix bugs
- narrate its process of writing the code
- preview pane to show p5.js canvas (w/ auto-refresh)
- Use whisper to listen to what I am saying
- Narrate the process, banter with me using TTS model?
- Fine-tune a model?
- RAG with nature of code book or other CT challenge examples?
- multi-modal so it can look at the canvas
My preference is for on device and fine-tuned model.
- Gemma
- transformers.js models
- llama (via ollama)
- localAI models