Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1110 in Layer 4

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.4586. Min Range: -3.4586

Max Act: 3.4586. Min Act: -0.1700

Data Index: 6642125 (Open Web Text)

Max Activating Token Index: 712

Text #1

Max Range: 3.4586. Min Range: -3.4586

Max Act: 3.4070. Min Act: -0.1700

Data Index: 4530104 (Open Web Text)

Max Activating Token Index: 457

Text #2

Max Range: 3.4586. Min Range: -3.4586

Max Act: 3.3574. Min Act: -0.1700

Data Index: 7875145 (Open Web Text)

Max Activating Token Index: 124

Text #3

Max Range: 3.4586. Min Range: -3.4586

Max Act: 3.3423. Min Act: -0.1700

Data Index: 4998195 (Open Web Text)

Max Activating Token Index: 805

Text #4

Max Range: 3.4586. Min Range: -3.4586

Max Act: 3.3319. Min Act: -0.1700

Data Index: 4542430 (Open Web Text)

Max Activating Token Index: 39

Text #5

Max Range: 3.4586. Min Range: -3.4586

Max Act: 3.3283. Min Act: -0.1700

Data Index: 4499005 (Open Web Text)

Max Activating Token Index: 380

Text #6

Max Range: 3.4586. Min Range: -3.4586

Max Act: 3.3008. Min Act: -0.1700

Data Index: 6316373 (Open Web Text)

Max Activating Token Index: 999

Text #7

Max Range: 3.4586. Min Range: -3.4586

Max Act: 3.3048. Min Act: -0.1700

Data Index: 2243058 (Open Web Text)

Max Activating Token Index: 397

Text #8

Max Range: 3.4586. Min Range: -3.4586

Max Act: 3.2998. Min Act: -0.1700

Data Index: 2888215 (Open Web Text)

Max Activating Token Index: 144

Text #9

Max Range: 3.4586. Min Range: -3.4586

Max Act: 3.2836. Min Act: -0.1700

Data Index: 8518345 (Open Web Text)

Max Activating Token Index: 978

Text #10

Max Range: 3.4586. Min Range: -3.4586

Max Act: 3.2939. Min Act: -0.1700

Data Index: 2425638 (Open Web Text)

Max Activating Token Index: 572

Text #11

Max Range: 3.4586. Min Range: -3.4586

Max Act: 3.2798. Min Act: -0.1700

Data Index: 386231 (Open Web Text)

Max Activating Token Index: 59

Text #12

Max Range: 3.4586. Min Range: -3.4586

Max Act: 3.2841. Min Act: -0.1700

Data Index: 1401185 (Open Web Text)

Max Activating Token Index: 289

Text #13

Max Range: 3.4586. Min Range: -3.4586

Max Act: 3.2467. Min Act: -0.1700

Data Index: 5171033 (Open Web Text)

Max Activating Token Index: 998

Text #14

Max Range: 3.4586. Min Range: -3.4586

Max Act: 3.2636. Min Act: -0.1700

Data Index: 4183530 (Open Web Text)

Max Activating Token Index: 376

Text #15

Max Range: 3.4586. Min Range: -3.4586

Max Act: 3.2603. Min Act: -0.1700

Data Index: 4736621 (Open Web Text)

Max Activating Token Index: 420

Text #16

Max Range: 3.4586. Min Range: -3.4586

Max Act: 3.2519. Min Act: -0.1700

Data Index: 5903954 (Open Web Text)

Max Activating Token Index: 663

Text #17

Max Range: 3.4586. Min Range: -3.4586

Max Act: 3.2621. Min Act: -0.1700

Data Index: 1430413 (Open Web Text)

Max Activating Token Index: 721

Text #18

Max Range: 3.4586. Min Range: -3.4586

Max Act: 3.2451. Min Act: -0.1700

Data Index: 6819246 (Open Web Text)

Max Activating Token Index: 191

Text #19

Max Range: 3.4586. Min Range: -3.4586

Max Act: 3.2445. Min Act: -0.1700

Data Index: 6258708 (Open Web Text)

Max Activating Token Index: 584