CoCalc Logo Icon
StoreFeaturesDocsShareSupportNewsAboutSign UpSign In
huggingface

Real-time collaboration for Jupyter Notebooks, Linux Terminals, LaTeX, VS Code, R IDE, and more,
all in one place. Commercial Alternative to JupyterHub.

GitHub Repository: huggingface/notebooks
Path: blob/main/course/en/chapter8/section3.ipynb
Views: 2555
Kernel: Unknown Kernel

Asking for help on the forums

Install the Transformers, Datasets, and Evaluate libraries to run this notebook.

!pip install datasets evaluate transformers[sentencepiece]
from transformers import AutoTokenizer, AutoModel model_checkpoint = "distilbert-base-uncased" tokenizer = AutoTokenizer.from_pretrained(model_checkpoint) model = AutoModel.from_pretrained(model_checkpoint)
text = """ Generation One is a retroactive term for the Transformers characters that appeared between 1984 and 1993. The Transformers began with the 1980s Japanese toy lines Micro Change and Diaclone. They presented robots able to transform into everyday vehicles, electronic items or weapons. Hasbro bought the Micro Change and Diaclone toys, and partnered with Takara. Marvel Comics was hired by Hasbro to create the backstory; editor-in-chief Jim Shooter wrote an overall story, and gave the task of creating the characthers to writer Dennis O'Neil. Unhappy with O'Neil's work (although O'Neil created the name "Optimus Prime"), Shooter chose Bob Budiansky to create the characters. The Transformers mecha were largely designed by Shōji Kawamori, the creator of the Japanese mecha anime franchise Macross (which was adapted into the Robotech franchise in North America). Kawamori came up with the idea of transforming mechs while working on the Diaclone and Macross franchises in the early 1980s (such as the VF-1 Valkyrie in Macross and Robotech), with his Diaclone mechs later providing the basis for Transformers. The primary concept of Generation One is that the heroic Optimus Prime, the villainous Megatron, and their finest soldiers crash land on pre-historic Earth in the Ark and the Nemesis before awakening in 1985, Cybertron hurtling through the Neutral zone as an effect of the war. The Marvel comic was originally part of the main Marvel Universe, with appearances from Spider-Man and Nick Fury, plus some cameos, as well as a visit to the Savage Land. The Transformers TV series began around the same time. Produced by Sunbow Productions and Marvel Productions, later Hasbro Productions, from the start it contradicted Budiansky's backstories. The TV series shows the Autobots looking for new energy sources, and crash landing as the Decepticons attack. Marvel interpreted the Autobots as destroying a rogue asteroid approaching Cybertron. Shockwave is loyal to Megatron in the TV series, keeping Cybertron in a stalemate during his absence, but in the comic book he attempts to take command of the Decepticons. The TV series would also differ wildly from the origins Budiansky had created for the Dinobots, the Decepticon turned Autobot Jetfire (known as Skyfire on TV), the Constructicons (who combine to form Devastator),[19][20] and Omega Supreme. The Marvel comic establishes early on that Prime wields the Creation Matrix, which gives life to machines. In the second season, the two-part episode The Key to Vector Sigma introduced the ancient Vector Sigma computer, which served the same original purpose as the Creation Matrix (giving life to Transformers), and its guardian Alpha Trion. """ inputs = tokenizer(text, return_tensors="pt") logits = model(**inputs).logits