Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2130 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.5301. Min Range: -3.5301

Max Act: 3.5301. Min Act: -0.1700

Data Index: 2965790 (Open Web Text)

Max Activating Token Index: 868

Text #1

Max Range: 3.5301. Min Range: -3.5301

Max Act: 3.2175. Min Act: -0.1700

Data Index: 2855118 (Open Web Text)

Max Activating Token Index: 664

Text #2

Max Range: 3.5301. Min Range: -3.5301

Max Act: 3.1387. Min Act: -0.1700

Data Index: 1219051 (Open Web Text)

Max Activating Token Index: 37

Text #3

Max Range: 3.5301. Min Range: -3.5301

Max Act: 3.0967. Min Act: -0.1700

Data Index: 4614798 (Open Web Text)

Max Activating Token Index: 446

Text #4

Max Range: 3.5301. Min Range: -3.5301

Max Act: 3.0903. Min Act: -0.1700

Data Index: 2830259 (Open Web Text)

Max Activating Token Index: 439

Text #5

Max Range: 3.5301. Min Range: -3.5301

Max Act: 3.0967. Min Act: -0.1700

Data Index: 8659335 (Open Web Text)

Max Activating Token Index: 310

Text #6

Max Range: 3.5301. Min Range: -3.5301

Max Act: 3.0362. Min Act: -0.1700

Data Index: 1737426 (Open Web Text)

Max Activating Token Index: 193

Text #7

Max Range: 3.5301. Min Range: -3.5301

Max Act: 3.0278. Min Act: -0.1700

Data Index: 5392370 (Open Web Text)

Max Activating Token Index: 515

Text #8

Max Range: 3.5301. Min Range: -3.5301

Max Act: 3.0541. Min Act: -0.1700

Data Index: 6539086 (Open Web Text)

Max Activating Token Index: 781

Text #9

Max Range: 3.5301. Min Range: -3.5301

Max Act: 3.0312. Min Act: -0.1700

Data Index: 1674592 (Open Web Text)

Max Activating Token Index: 448

Text #10

Max Range: 3.5301. Min Range: -3.5301

Max Act: 3.0298. Min Act: -0.1700

Data Index: 5413334 (Open Web Text)

Max Activating Token Index: 546

Text #11

Max Range: 3.5301. Min Range: -3.5301

Max Act: 3.0726. Min Act: -0.1700

Data Index: 7170076 (Open Web Text)

Max Activating Token Index: 748

Text #12

Max Range: 3.5301. Min Range: -3.5301

Max Act: 3.0318. Min Act: -0.1700

Data Index: 6062123 (Open Web Text)

Max Activating Token Index: 55

Text #13

Max Range: 3.5301. Min Range: -3.5301

Max Act: 3.0277. Min Act: -0.1700

Data Index: 7036415 (Open Web Text)

Max Activating Token Index: 512

Text #14

Max Range: 3.5301. Min Range: -3.5301

Max Act: 2.9952. Min Act: -0.1700

Data Index: 3725496 (Open Web Text)

Max Activating Token Index: 962

Text #15

Max Range: 3.5301. Min Range: -3.5301

Max Act: 2.9953. Min Act: -0.1700

Data Index: 708205 (Open Web Text)

Max Activating Token Index: 495

Text #16

Max Range: 3.5301. Min Range: -3.5301

Max Act: 3.0020. Min Act: -0.1700

Data Index: 6547594 (Open Web Text)

Max Activating Token Index: 891

Text #17

Max Range: 3.5301. Min Range: -3.5301

Max Act: 3.0100. Min Act: -0.1700

Data Index: 3775819 (Open Web Text)

Max Activating Token Index: 889

Text #18

Max Range: 3.5301. Min Range: -3.5301

Max Act: 2.9789. Min Act: -0.1700

Data Index: 4773790 (Open Web Text)

Max Activating Token Index: 346

Text #19

Max Range: 3.5301. Min Range: -3.5301

Max Act: 2.9580. Min Act: -0.1700

Data Index: 3259702 (Open Web Text)

Max Activating Token Index: 200