{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": "# fak \u2014 Tier 2: the fused in-kernel model (neocloud notebook)\n\nThis is **Notebook B** \u2014 the GPU-deep companion to `fak-quickstart.ipynb`. It runs the\ndeepest fusion: the model runs **inside the kernel's address space**, decoding over a\nkernel-owned KV cache, reachable via `/v1/fak/syscall`. It then fronts a **bigger** model\nand exposes a **stable endpoint** an external client can reach \u2014 the two things a free,\nno-ingress Colab can't do.\n\nHost it on a neocloud Jupyter with a real terminal and port ingress \u2014 **Lightning AI Studio**\nor a **RunPod / Vast** GPU pod (the `[VERIFIED]` single-GPU rows in\n`PLAN-cloud-neocloud-rightsizing-2026-06-20.md`).\n\n| Section | What it shows | Needs |\n|---|---|---|\n| **2a \u2014 synthetic in-kernel** | the real prefill+decode loop over a kernel-owned KV cache | **CPU ok** |\n| **2b \u2014 real SmolLM2-135M** | the same path on weights exported from HuggingFace | **CPU ok** (Python/torch) |\n| **bigger model + endpoint** | `qwen2.5:14b` behind `fak serve` on a 24 GB card, exposed publicly | **GPU** (24 GB) |\n\n> Honest scope (matches `CLAIMS.md`): the in-kernel paths are proven correct at the\n> tensor layer against a HuggingFace oracle \u2014 they make model state first-class kernel-owned\n> state; they are **not** a production chat-quality serving engine. For practical chat, the\n> bigger-model section uses Tier 1 (front Ollama).\n\n> _Generated by `tools/gen_notebooks.py` \u2014 edit the generator, not this `.ipynb`. CI guards drift + stale references via `--check`._"
  },
  {
   "cell_type": "code",
   "metadata": {},
   "execution_count": null,
   "outputs": [],
   "source": "# --- setup: work dir, repo coords, GPU ---\nimport os, shutil, subprocess, sys, time, json, urllib.request\n\nWORK = os.environ.get(\"FAK_WORK\") or (\"/content\" if os.path.isdir(\"/content\")\n        else \"/kaggle/working\" if os.path.isdir(\"/kaggle/working\") else \"/tmp/fak-demo\")\nos.makedirs(WORK, exist_ok=True); os.chdir(WORK)\n\nREPO   = os.environ.get(\"FAK_REPO\", \"anthony-chaudhary/fak\")\nBRANCH = os.environ.get(\"FAK_BRANCH\", \"main\")  # pin a release tag here, e.g. v0.30.0\n\nHAS_GPU = shutil.which(\"nvidia-smi\") is not None and \\\n          subprocess.run([\"nvidia-smi\"], capture_output=True).returncode == 0\nprint(\"work dir :\", WORK)\nprint(f\"repo     : {REPO}@{BRANCH}\")\nprint(\"GPU      :\", \"yes\" if HAS_GPU else \"no \u2014 Tier 2 in-kernel still runs on CPU; the bigger-model section is skipped\")"
  },
  {
   "cell_type": "code",
   "metadata": {},
   "execution_count": null,
   "outputs": [],
   "source": "# --- get a fresh `fak` binary plus the examples/testdata the demos use.\n#     The Go module is the repository ROOT (go.mod), so the clone dir *is* the build dir.\n#     By default an existing clone is refreshed to FAK_BRANCH. Set FAK_REFRESH=0 to reuse it.\nimport urllib.request, tarfile, platform, re\n\ndef sh(cmd, timeout=900):\n    print(\"$\", cmd)\n    subprocess.run(cmd, shell=True, check=True, timeout=timeout)\n\ndef go_ok():\n    try:\n        out = subprocess.run([\"go\",\"version\"], capture_output=True, text=True, check=True).stdout\n        m = re.search(r\"go(\\d+)\\.(\\d+)\", out)\n        return bool(m) and (int(m.group(1)), int(m.group(2))) >= (1, 26)\n    except Exception:\n        return False\n\ndef latest_go_archive():\n    arch = platform.machine().lower()\n    goarch = {\"x86_64\":\"amd64\", \"amd64\":\"amd64\", \"aarch64\":\"arm64\", \"arm64\":\"arm64\"}.get(arch)\n    if sys.platform.startswith(\"linux\") and goarch:\n        rels = json.load(urllib.request.urlopen(\"https://go.dev/dl/?mode=json\", timeout=30))\n        for rel in rels:\n            m = re.match(r\"go(\\d+)\\.(\\d+)\", rel.get(\"version\", \"\"))\n            if not m or (int(m.group(1)), int(m.group(2))) < (1, 26):\n                continue\n            for f in rel.get(\"files\", []):\n                if f.get(\"kind\") == \"archive\" and f.get(\"os\") == \"linux\" and f.get(\"arch\") == goarch:\n                    return rel[\"version\"], f[\"filename\"]\n    return \"go1.26.3\", \"go1.26.3.linux-amd64.tar.gz\"\n\nif not go_ok():\n    GOV, GO = latest_go_archive()\n    print(\"installing\", GOV, \"...\")\n    urllib.request.urlretrieve(\"https://go.dev/dl/\" + GO, \"/tmp/\" + GO)\n    shutil.rmtree(\"/usr/local/go\", ignore_errors=True)\n    with tarfile.open(\"/tmp/\" + GO) as t: t.extractall(\"/usr/local\")\nos.environ[\"PATH\"] = \"/usr/local/go/bin:\" + os.environ[\"PATH\"]\nprint(subprocess.run([\"go\",\"version\"], capture_output=True, text=True).stdout.strip())\n\n# clone the public repo (anonymous; no token needed), then refresh to the requested ref\nFAK_DIR = os.path.join(WORK, \"fak\")\nif not os.path.isdir(os.path.join(FAK_DIR, \".git\")):\n    print(f\"$ git clone --depth 1 -b {BRANCH} https://github.com/{REPO}.git  ->  {FAK_DIR}\")\n    subprocess.run([\"git\",\"clone\",\"--depth\",\"1\",\"-b\",BRANCH,\n                    f\"https://github.com/{REPO}.git\", FAK_DIR], check=True)\nelse:\n    print(\"clone exists:\", FAK_DIR)\n    if os.environ.get(\"FAK_REFRESH\", \"1\") != \"0\":\n        sh(f'git -C \"{FAK_DIR}\" fetch --depth 1 origin \"{BRANCH}\"')\n        sh(f'git -C \"{FAK_DIR}\" checkout -q --detach FETCH_HEAD')\n    else:\n        print(\"FAK_REFRESH=0, reusing the existing checkout without fetch\")\n\nstamp = subprocess.run([\"git\",\"-C\",FAK_DIR,\"log\",\"-1\",\"--format=%h %cI %s\"],\n                       capture_output=True, text=True, check=True).stdout.strip()\nprint(\"source  :\", stamp)\n\n# the module is the repo root, so build right here\nFAK_EXE = \"fak.exe\" if os.name == \"nt\" else \"fak\"\nFAK = os.path.join(FAK_DIR, FAK_EXE)\nsh(f'cd \"{FAK_DIR}\" && go build -o \"{FAK_EXE}\" ./cmd/fak')\nprint(\"\\nfak ->\", FAK)\nprint(subprocess.run([FAK,\"version\"], capture_output=True, text=True).stdout.strip() or \"built ok\")"
  },
  {
   "cell_type": "code",
   "metadata": {},
   "execution_count": null,
   "outputs": [],
   "source": "# --- helpers: (re)start a fak serve on a port, and call the in-kernel syscall ---\ndef serve(args, port=8137, env=None):\n    subprocess.run('pkill -f \"fak serve\" 2>/dev/null; sleep 1', shell=True)\n    e = dict(os.environ)\n    if env: e.update(env)\n    p = subprocess.Popen([FAK,\"serve\",\"--addr\",f\"127.0.0.1:{port}\",*args],\n                         cwd=FAK_DIR, env=e, stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL)\n    for _ in range(90):\n        try:\n            with urllib.request.urlopen(f\"http://127.0.0.1:{port}/healthz\", timeout=2) as r:\n                print(\"fak serve up:\", r.read().decode()); return p\n        except Exception: time.sleep(1)\n    print(\"did not bind on\", port); return p\n\ndef syscall(tool, arguments, port=8137):\n    # NB: the fak-native wire key is \"arguments\" (NOT \"args\" \u2014 unknown keys are dropped)\n    req = urllib.request.Request(f\"http://127.0.0.1:{port}/v1/fak/syscall\",\n        data=json.dumps({\"tool\":tool,\"arguments\":arguments}).encode(),\n        headers={\"Content-Type\":\"application/json\"})\n    return json.load(urllib.request.urlopen(req, timeout=120))\n\nprint(\"helpers ready\")"
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": "## 2a \u2014 synthetic in-kernel checkpoint (instant, CPU ok)\n\n`--engine inkernel` with no weights runs a small deterministic synthetic checkpoint. The\ntokens are meaningless (random-init weights) \u2014 what it proves is the **real prefill+decode\nloop over the kernel-owned KV cache**, dispatched through the adjudicator."
  },
  {
   "cell_type": "code",
   "metadata": {},
   "execution_count": null,
   "outputs": [],
   "source": "serve([\"--engine\",\"inkernel\",\"--model\",\"smollm2-inkernel\"], port=8137)\nout = syscall(\"read_file\", {\"path\":\"notes.txt\"}, port=8137)\nprint(\"verdict :\", out.get(\"verdict\"))\nprint(\"meta    :\", out.get(\"result\", {}).get(\"meta\"))\nprint(\"content :\", out.get(\"result\", {}).get(\"content\", \"\")[:200])"
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": "## 2b \u2014 real SmolLM2-135M weights (CPU torch is enough)\n\n`scripts/fetch-model.sh` makes a Python venv, installs torch/transformers (CPU is fine),\ndownloads `HuggingFaceTB/SmolLM2-135M-Instruct`, and exports it into\n`internal/model/.cache/smollm2-135m`. First run needs network to HuggingFace; it takes a\nfew minutes. `FAK_MODEL_DIR` is what selects the real weights."
  },
  {
   "cell_type": "code",
   "metadata": {},
   "execution_count": null,
   "outputs": [],
   "source": "try:\n    print(\"exporting real weights (one-time, ~mins) ...\")\n    subprocess.run(f'cd \"{FAK_DIR}\" && ./scripts/fetch-model.sh', shell=True, check=True)\n    MODEL_DIR = os.path.join(FAK_DIR, \"internal/model/.cache/smollm2-135m\")\n    serve([\"--engine\",\"inkernel\",\"--model\",\"smollm2-135m\"], port=8137,\n          env={\"FAK_MODEL_DIR\": MODEL_DIR})\n    out = syscall(\"read_file\", {\"path\":\"notes.txt\"}, port=8137)\n    print(\"verdict :\", out.get(\"verdict\"))\n    print(\"meta    :\", out.get(\"result\", {}).get(\"meta\"))\nexcept subprocess.CalledProcessError as e:\n    print(\"fetch-model failed (needs Python 3.10+ and network to HuggingFace):\", e)\n    print(\"the synthetic path in 2a already exercised the in-kernel decode loop.\")"
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": "## Bigger model + a stable endpoint (needs a 24 GB GPU)\n\nA free Colab T4 (16 GB) tops out around a 7B model and has **no inbound port**. On a\nneocloud 24 GB card you can front `qwen2.5:14b` (~9 GB) through the kernel \u2014 the same shape\nas the live GCP demo, but reader-owned \u2014 and expose it so an external client (or Claude\nCode) can point at it."
  },
  {
   "cell_type": "code",
   "metadata": {},
   "execution_count": null,
   "outputs": [],
   "source": "if not HAS_GPU:\n    print(\"no GPU \u2014 skipping. (Tier 2 in-kernel above ran on CPU.)\")\nelse:\n    MODEL = os.environ.get(\"FAK_MODEL\", \"qwen2.5:14b\")  # ~9 GB, fits a 24 GB card\n    if shutil.which(\"ollama\") is None:\n        subprocess.run(\"curl -fsSL https://ollama.com/install.sh | sh\", shell=True, check=True)\n    subprocess.Popen([\"ollama\",\"serve\"], stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL)\n    for _ in range(60):\n        try: urllib.request.urlopen(\"http://127.0.0.1:11434/api/tags\", timeout=2); break\n        except Exception: time.sleep(1)\n    print(\"pulling\", MODEL, \"(one-time, cached) ...\")\n    subprocess.run([\"ollama\",\"pull\",MODEL], check=True)\n    serve([\"--base-url\",\"http://127.0.0.1:11434/v1\",\"--model\",MODEL], port=8080)\n\n    subprocess.run([sys.executable,\"-m\",\"pip\",\"install\",\"-q\",\"openai\"], check=True)\n    from openai import OpenAI\n    client = OpenAI(base_url=\"http://127.0.0.1:8080/v1\", api_key=\"not-needed\")\n    r = client.chat.completions.create(model=MODEL, temperature=0.3,\n        messages=[{\"role\":\"user\",\"content\":\"In one sentence, what is a syscall?\"}])\n    print(\"\\nmodel (through the kernel):\", r.choices[0].message.content)"
  },
  {
   "cell_type": "code",
   "metadata": {},
   "execution_count": null,
   "outputs": [],
   "source": "# Expose :8080 publicly. On Lightning AI: use the Studio \"expose port\" control.\n# On RunPod: the pod's proxy URL is https://<POD_ID>-8080.proxy.runpod.net .\n# Portable fallback (no signup) \u2014 a cloudflared quick tunnel; watch for the trycloudflare URL:\nif HAS_GPU:\n    try:\n        subprocess.run(\n            \"wget -q https://github.com/cloudflare/cloudflared/releases/latest/download/\"\n            \"cloudflared-linux-amd64 -O /usr/local/bin/cloudflared && chmod +x /usr/local/bin/cloudflared\",\n            shell=True, check=True)\n        print(\"starting cloudflared quick tunnel to :8080 \u2014 the https://*.trycloudflare.com\")\n        print(\"URL will print below; point any OpenAI client at  <that-url>/v1\")\n        subprocess.Popen([\"cloudflared\",\"tunnel\",\"--url\",\"http://localhost:8080\"])\n        time.sleep(8)\n    except Exception as e:\n        print(\"tunnel optional \u2014 on Lightning/RunPod use the platform's port-expose instead:\", e)"
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": "## Where to go next\n\n- **Why this is the deep fusion:** `docs/explainers/addressable-kv-cache.md` (mid-run causal\n  span eviction, bit-exact `max|\u0394| = 0`).\n- **The expert smoke** \u2014 Qwen3.6-27B through fak's own GGUF\u2192Q8 path (`cmd/fakchat`):\n  `GETTING-STARTED.md` \u00a74c\u20134d.\n- **The cost/hardware tiers** behind this notebook: `PLAN-cloud-neocloud-rightsizing-2026-06-20.md`.\n- **What's real vs. simulated vs. stub:** `CLAIMS.md`.\n\n*Cleanup:* `!pkill -f \"fak serve\"; pkill -f ollama; pkill -f cloudflared`"
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "name": "python"
  },
  "colab": {
   "provenance": []
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}
