Hacker News Re-Imagined

Reverse Engineering Google Colab

  • 155 points
  • 11 days ago

  • @arjvik
  • Created a post

Reverse Engineering Google Colab


@a-dub 11 days

Replying to @arjvik 🎙

https://github.com/singhsidhukuldeep/Google-Colab-Shell

pops a terminal inline in the colab notbook on the backing vm. super useful if you get tired of having to shell execute all the time via the cell interface.

Reply


@BrianHenryIE 11 days

Replying to @arjvik 🎙

There's an active effort to (again) implement Swift on Colab:

https://github.com/philipturner/swift-colab/

Reply


@tehsauce 11 days

Replying to @arjvik 🎙

Make some of its features can be pulled into jupyterlab!

Reply


@cperry 11 days

Replying to @arjvik 🎙

Impressive work.

Just came here to note that we read all of our in-product feedback submissions as well as GitHub issues: https://github.com/googlecolab/colabtools/issues

If you've got feature requests or encounter bugs we appreciate you filing!

Reply


@sillysaurusx 11 days

Replying to @arjvik 🎙

It’s about a bazillion times easier to reverse engineer colab if you just SSH into it. You can set up a reverse proxy. I used ngrok back in the day, but maybe they blocked it.

The most interesting thing was a custom binary that mounts your Google drive as a folder. I was able to copy it off colab and use it on my own Linux boxes, which was handy in a “oh neat, lookie there” kind of way. I assume it’ll break whenever they update their api, but you’d still be able to just grab the new binary from a random colab instance.

There’s also a custom script they run to set up everything, using Node. It spawns a bunch of stuff that I’ve forgotten. (It was 2019 when I was poking around, and a pandemic has a nice way of wiping one’s memory of ye olden hacking days. Still a bit sad I never got to go to the tensorflow conference.)

Anyway just ssh in and ls -la / and you’ll see one or two interesting folders. You can rsync them down to your box and examine at your leisure.

Reply


@minimaxir 11 days

Replying to @arjvik 🎙

> However, it's incredibly difficult to harness the compute power of Colab for anything beyond Jupyter notebooks. For Machine Learning engineers that want to productionize their models and bring them out of the notebook stage, this is a particularly relevant issue; notebooks, while perfect for exploration, don't play well with more advanced MLOps tools that codify the training process into a formal pipeline.

That isn't what Colab is intended for. Google has better and more productive tools for companies who can fit the bill, which is getting cheaper over time.

AI Notebooks behave the same in practice as Google Colab with one-click one/off for model testing + JupyterLab. If you want to minimize costs via spot instances, you can deploy a Compute Engine with the Deep Learning VM image, which also includes a running JupyterLab on launch if need to use that workflow, and also saves time by including your framework of choice. A spot VM with a T4 GPU is about $0.18/hour.

Reply


About Us

site design / logo © 2022 Box Piper