Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 214 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.7013. Min Range: -3.7013

Max Act: 3.7013. Min Act: -0.1700

Data Index: 5131860 (Open Web Text)

Max Activating Token Index: 852

Text #1

Max Range: 3.7013. Min Range: -3.7013

Max Act: 3.6608. Min Act: -0.1700

Data Index: 5476055 (Open Web Text)

Max Activating Token Index: 704

Text #2

Max Range: 3.7013. Min Range: -3.7013

Max Act: 3.5624. Min Act: -0.1700

Data Index: 1680281 (Open Web Text)

Max Activating Token Index: 915

Text #3

Max Range: 3.7013. Min Range: -3.7013

Max Act: 3.5655. Min Act: -0.1700

Data Index: 5476073 (Open Web Text)

Max Activating Token Index: 554

Text #4

Max Range: 3.7013. Min Range: -3.7013

Max Act: 3.5125. Min Act: -0.1700

Data Index: 5627014 (Open Web Text)

Max Activating Token Index: 89

Text #5

Max Range: 3.7013. Min Range: -3.7013

Max Act: 3.5024. Min Act: -0.1700

Data Index: 8439541 (Open Web Text)

Max Activating Token Index: 887

Text #6

Max Range: 3.7013. Min Range: -3.7013

Max Act: 3.4858. Min Act: -0.1700

Data Index: 7799738 (Open Web Text)

Max Activating Token Index: 354

Text #7

Max Range: 3.7013. Min Range: -3.7013

Max Act: 3.5200. Min Act: -0.1700

Data Index: 6327787 (Open Web Text)

Max Activating Token Index: 339

Text #8

Max Range: 3.7013. Min Range: -3.7013

Max Act: 3.4755. Min Act: -0.1700

Data Index: 5850358 (Open Web Text)

Max Activating Token Index: 466

Text #9

Max Range: 3.7013. Min Range: -3.7013

Max Act: 3.4807. Min Act: -0.1700

Data Index: 8542939 (Open Web Text)

Max Activating Token Index: 736

Text #10

Max Range: 3.7013. Min Range: -3.7013

Max Act: 3.4542. Min Act: -0.1700

Data Index: 1680286 (Open Web Text)

Max Activating Token Index: 181

Text #11

Max Range: 3.7013. Min Range: -3.7013

Max Act: 3.4605. Min Act: -0.1700

Data Index: 6240059 (Open Web Text)

Max Activating Token Index: 629

Text #12

Max Range: 3.7013. Min Range: -3.7013

Max Act: 3.4454. Min Act: -0.1700

Data Index: 3384243 (Open Web Text)

Max Activating Token Index: 246

Text #13

Max Range: 3.7013. Min Range: -3.7013

Max Act: 3.4491. Min Act: -0.1700

Data Index: 5378940 (Open Web Text)

Max Activating Token Index: 326

Text #14

Max Range: 3.7013. Min Range: -3.7013

Max Act: 3.4279. Min Act: -0.1700

Data Index: 5877937 (Open Web Text)

Max Activating Token Index: 385

Text #15

Max Range: 3.7013. Min Range: -3.7013

Max Act: 3.4332. Min Act: -0.1700

Data Index: 7013518 (Open Web Text)

Max Activating Token Index: 180

Text #16

Max Range: 3.7013. Min Range: -3.7013

Max Act: 3.4173. Min Act: -0.1700

Data Index: 5984397 (Open Web Text)

Max Activating Token Index: 742

Text #17

Max Range: 3.7013. Min Range: -3.7013

Max Act: 3.4219. Min Act: -0.1700

Data Index: 812179 (Open Web Text)

Max Activating Token Index: 758

Text #18

Max Range: 3.7013. Min Range: -3.7013

Max Act: 3.4132. Min Act: -0.1700

Data Index: 4720588 (Open Web Text)

Max Activating Token Index: 36

Text #19

Max Range: 3.7013. Min Range: -3.7013

Max Act: 3.4026. Min Act: -0.1700

Data Index: 2642244 (Open Web Text)

Max Activating Token Index: 922