# Getting a course from Canvas > From getsimeon.com/docs/getting-a-course-from-canvas (28 October 2026). A write-up of how Simeon is built, for a coding agent (Claude Code, Codex, Cursor…) to use as a reference when building something like it. File paths are Simeon's own: take the ideas, not the paths. ## The idea in one paragraph Simeon doesn't scrape Canvas's web pages, and it never sees a password. It reads a course through **Canvas's own REST API** (`/api/v1/…`), from inside a Canvas tab where the student is already signed in, so every request is a plain same-origin `fetch` carrying only their session cookie: no API token, no developer key, no OAuth app for a school to approve. It is **read-only and polite**: one function, `cfetch`, sends every request, never more than three at a time, and backs off when Canvas says to. The result is one zip, `Simeon - .zip`: the course as Markdown notes, every file as it was uploaded, and `course.json`, which describes the course as data. There is **one copy of that logic**, `extension/mapper.js`, and three hosts run it unchanged: a Chrome extension, the Mac app and the Windows app. > **Figure:** The Chrome extension, the Mac app and the Windows app each run the same mapper.js inside a signed-in Canvas page, which calls Canvas's REST API and produces one zip. Three hosts, one export. Each host only has to get mapper.js into a signed-in Canvas page and carry its messages and its zip; everything that talks to Canvas is the same code. --- ## 1. Why it runs inside Canvas Two limits shape the design: - **Simeon's page can't call Canvas.** It is a local `file://` page (or getsimeon.com): another origin, with none of the student's Canvas cookies, and Canvas doesn't allow cross-origin reads of its API. - **A `file://` page can't message a Chrome extension at all.** So the export runs **on Canvas's own origin**, in a page where the student is signed in through their school's usual sign-in (SSO, MFA, whatever it is). Every API call is `fetch(url, { credentials: 'include' })`. Nothing about the session is stored by Simeon: in the apps, the web view's own data store keeps it between launches, as a browser would. This also sidesteps the usual blocker for Canvas tools. Personal access tokens are often switched off by schools, and an OAuth developer key needs the school's admin. A student's own session needs nobody's permission and can only read what the student could already open. ## 2. Getting messages in and out ### In Chrome ``` Simeon page ──postMessage──▶ bridge.js ──runtime message──▶ background.js │ finds a Canvas tab, or opens ▼ one in the background mapper.js, in the Canvas tab ◀── bridge.js ◀── background.js ◀──── events: progress … done | error zip ──▶ Downloads ``` - **`bridge.js`** is a content script on Simeon's page. Because it is matched on every local file, it does nothing unless the page has ``. It shares the page's window, so the two talk with `postMessage`. - **`background.js`** (the service worker) finds a Canvas tab or opens one in the background, and pings it every 500 ms for up to about 20 seconds. A tab opened before the extension was installed never got the content script, so after two failed pings on a loaded Canvas tab it injects it with `chrome.scripting`; `mapper.js` guards itself so it never runs twice. If Canvas never answers, it's nearly always the school's sign-in page, so that tab is brought to the front. - **`mapper.js`** answers an export **at once** (`{ ok, started }`) and then reports through separate event messages addressed to the tab that asked. Nothing waits on one long-lived message: a course with its files takes minutes, longer than an MV3 service worker stays awake, and each event wakes it again. ### In the apps The apps have no extension, so a small `canvas-bridge.js` stands in for the two things a content script expects: 1. **`chrome.runtime`.** A fake `sendMessage` posts events to the app; the app asks mapper.js things by calling its listener exactly as Chrome would. 2. **A browser that saves ``.** mapper.js hands over the finished zip by clicking a download link to a blob. The bridge wraps `URL.createObjectURL` to remember the blobs it made and `HTMLAnchorElement.click` to catch a click on one of those, then sends the bytes to the app as base64 in **3 MB slices**, which the app reassembles and writes atomically. Every other click on the page behaves as usual. On the Mac the scripts run in an **isolated content world**, so none of these overrides are visible to Canvas's own scripts. The web view re-injects after each navigation that lands back on the Canvas host (a new page is a new JavaScript world), opens sign-in pop-ups ("Sign in with Microsoft") in the same view, and is told not to throttle itself when off screen, so the export carries on at full speed once its sheet closes. Windows needed one more piece. A Canvas file download redirects to Amazon S3, and WebView2 (Chromium) blocks the page from reading that cross-origin response, which Chrome lets the extension do and the Mac's WebKit doesn't enforce. So the Windows bridge wraps `fetch` for same-host `GET …/files/:id/download` only, and the app fetches the file itself: it sends the window's Canvas cookies **to the Canvas host only**, follows the HTTPS redirects without them, and streams the bytes back in 6 MB pieces. Status codes and the rate-limit header are passed through, so the back-off below still works. ## 3. Being polite: `cfetch` Every request goes through one function: - **At most three in flight** across the whole export. A freed slot is handed straight to the next waiter. - **Backs off when told to.** A `429`, or a `403` whose body says "rate limit", waits 1, 2, 4, 8, 16 seconds (capped at 30) and tries again, up to five times. - **Watches its budget.** When `X-Rate-Limit-Remaining` drops below 150 it pauses a second; below 50, three. - **Read-only.** Every call is a `GET`. Nothing on Canvas is ever posted, submitted or changed. Lists are fetched with `per_page=100`, following the `Link: <…>; rel="next"` header to the end. A page that isn't OK ends the list quietly, so an endpoint a course has switched off gives an empty list, not a failed export. ## 4. What it reads First the course, with `include[]=syllabus_body`, `term` and `teachers`. Then, three on the wire at a time: | Endpoint | For | |---|---| | `/courses/:id/modules?include[]=items` | the modules and their items, in order | | `/courses/:id/assignments` | descriptions, attachments, rubrics | | `/courses/:id/pages` | the page list (often switched off: see below) | | `/courses/:id/discussion_topics` (and `only_announcements=true`) | discussions and announcements | | `/courses/:id/files` and `/folders` | the file list and its folders (often switched off) | | `/courses/:id/assignment_groups?include[]=assignments` | how the course is graded | | `/calendar_events?context_codes[]=course_:id&type=event` | the exam date and room, review sessions | | `/courses/:id/users?enrollment_type[]=teacher&enrollment_type[]=ta` | who is staff | **Pages when the Pages tab is off.** Many instructors hide the Pages tab, and the list 404s, but each page can still be read by its slug. The export takes the union of the list and every module item of type `Page`, and fetches each. **Discussions.** A topic listed with an empty body is fetched again on its own. Every topic's replies come from `/discussion_topics/:id/view`, which returns the whole reply tree and its participants in one call. ## 5. Finding every file This is where most of the work is. Instructors often switch the Files tab off too (`/files` answers 403), and slides then live only in modules, attachments and links. So file ids are gathered from **every source** and unioned: 1. the Files API, when it answers 2. module items of type `File` 3. attachments on assignments, discussions, announcements, and on individual replies 4. **links in every HTML body**: the syllabus, assignment descriptions, pages, discussion and announcement messages, every reply, and each module item's external URL The link pass (`extractFileRefs`) walks `a`, `img`, `source`, `iframe`, `embed` and `object` elements for `/courses//files/` and `/files/`, then runs a regex for plain-text URLs the DOM walk missed. It keeps **the course the link points into and the link's text**, not just the id. That matters because instructors paste pages from older instances of a course, so a link often points at a file in another course. Each id without metadata is then looked up in the course its link points into, then in this course, then globally (`/files/:id`). If none answers, the student has no access: the file lives in a course they aren't in, and nothing, not even the student clicking the link, can download it. It is kept as **restricted**, with its name and a link to open it in Canvas, rather than silently dropped. **Downloading** tries, in order: the signed CDN link from the metadata, then `…/files/:id/download?download_frd=1` under the linked course, this course, and no course. Each file is fingerprinted with SHA-256 (`crypto.subtle`), and its path in the zip mirrors Canvas's own folders. **Files hosted off Canvas.** Pages often link straight to publisher or library PDFs. Links that aren't on Canvas and end in a document extension are collected, and fetched **outside the page** (the extension's service worker, or the app) **with no cookies**, so the student's session never reaches a third-party site. Each is capped at 40 MB; bigger ones stay a link. ## 6. Names stay out By default the export writes a discussion reply's author as **The lecturer** (anyone on the course's teacher or TA lists) or **Student A, B…**, lettered per thread so a conversation still reads. `course.json` says how many teach the course, not who. Every name the export saw goes in `_names.json`, and Simeon's engine replaces those names in every text before any AI reads it, then deletes the file. A student can turn this off in the app's settings. ## 7. What comes out HTML bodies become Markdown through a small DOM walker (headings, lists, tables, code, links, images; scripts and styles dropped), and each note starts with YAML frontmatter (`type`, `course`, `url`, dates) for the engine to read. ``` Simeon - / Course Index.md every module, assignment, page and topic, linked Map.md the modules as a Mermaid diagram Syllabus.md · Calendar.md · Grading.md Modules/ · Assignments/ · Pages/ · Discussions/ · Announcements/ Files/ every downloaded file, in Canvas's folders, with Files Index.md Files/External/ the files hosted off Canvas course.json the course as data ``` **`course.json`** saves a build from working the course out of file names. It lists each module in order and, for each file, its module and position (from a module item, or from the page or assignment that linked it, when that is in a module), dates, size, fingerprint, and a guessed **kind**: lecture, seminar, exam, solutions, reading, case, assignment, course info or other, matched on the name, then the folder, then the module, in English, Swedish and Dutch. It marks **duplicates** (the same bytes twice), **older versions** (the same name without "v2", "final", "updated"…) and pairs each **exam with its solutions** by name. All of those are guesses, and the file says so; the agent that builds the plan is told to check a file before trusting what it's called. The zip is built with JSZip as a **blob** and handed over with an object URL. Not base64 through `chrome.downloads`: extension messages cap at about 64 MB, and a course with its slides can be far bigger. ## 8. When something goes missing Every export logs structured objects in the Canvas tab's console: what it found per source (`Simeon file discovery: {…}`, with files per source, links per body type, cross-course links, restricted and external), and download summaries. When files are missing, that object usually says why. All zeros means the material is in an external tool (a video platform, SCORM) that the Canvas API doesn't expose. On Windows each step is also written, with its time, to a log beside the app's settings, which shows which step was slow. The whole chain is tested end to end: the real extension loaded into headless Chrome over the DevTools pipe, against a fake Canvas served over local HTTPS, with a Canvas tab opened before the extension, no Canvas tab at all, and a signed-out Canvas. ## Reusing it: a recipe For any tool that needs a student's (or employee's) data from a web app they're already signed in to: 1. **Run inside the signed-in origin** and call the app's own API with the session cookie. No tokens to issue, no admin to ask, and access exactly equal to what the user can already see. 2. **Put every request through one polite function**: a global concurrency cap, exponential back-off on 429 (and on whatever the app returns instead), and a pause when the rate-limit header runs low. Read-only. 3. **Don't trust the obvious endpoint.** Union every place an item can be referenced from, keep where each reference points, and resolve it from there first. List what you can't reach instead of dropping it. 4. **Fetch third-party links outside the page, with no cookies.** 5. **Write one export and adapt the hosts.** A small shim for messaging and file saving let the same script run in an extension, WebKit and WebView2 without a fork. 6. **Answer at once, report as events.** Long jobs outlive a single message. 7. **Ship data alongside documents** (`course.json`): what you learn while exporting is cheap to write down and expensive to guess later. Mark guesses as guesses. ### Where to look in the code | What | File | |---|---| | The whole export | `extension/mapper.js` | | Chrome: page ↔ extension ↔ Canvas tab | `extension/bridge.js`, `extension/background.js` | | Mac: the web view and the zip | `local/mac-app/Sources/CanvasExport.swift`, `Resources/canvas-bridge.js` | | Windows: the window, file downloads | `desktop/src-tauri/src/canvas.rs`, `scripts/canvas-bridge.js` | | Reading the zip | `local/course_context/ingest.py` | | Taking names out | `local/course_context/names.py` | | End-to-end test | `tests/test_extension_e2e.js` |