Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 889 in Layer 4

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.6828. Min Range: -4.6828

Max Act: 4.6828. Min Act: -0.1700

Data Index: 962325 (Open Web Text)

Max Activating Token Index: 669

Text #1

Max Range: 4.6828. Min Range: -4.6828

Max Act: 4.4890. Min Act: -0.1700

Data Index: 1838595 (Open Web Text)

Max Activating Token Index: 546

Text #2

Max Range: 4.6828. Min Range: -4.6828

Max Act: 4.2721. Min Act: -0.1700

Data Index: 4844854 (Open Web Text)

Max Activating Token Index: 440

Text #3

Max Range: 4.6828. Min Range: -4.6828

Max Act: 4.2341. Min Act: -0.1700

Data Index: 6060998 (Open Web Text)

Max Activating Token Index: 131

Text #4

Max Range: 4.6828. Min Range: -4.6828

Max Act: 4.2340. Min Act: -0.1700

Data Index: 7135422 (Open Web Text)

Max Activating Token Index: 398

Text #5

Max Range: 4.6828. Min Range: -4.6828

Max Act: 4.2097. Min Act: -0.1700

Data Index: 934349 (Open Web Text)

Max Activating Token Index: 91

Text #6

Max Range: 4.6828. Min Range: -4.6828

Max Act: 4.1628. Min Act: -0.1700

Data Index: 5125399 (Open Web Text)

Max Activating Token Index: 357

Text #7

Max Range: 4.6828. Min Range: -4.6828

Max Act: 4.1310. Min Act: -0.1700

Data Index: 8783702 (Open Web Text)

Max Activating Token Index: 920

Text #8

Max Range: 4.6828. Min Range: -4.6828

Max Act: 3.9475. Min Act: -0.1700

Data Index: 5530617 (Open Web Text)

Max Activating Token Index: 566

Text #9

Max Range: 4.6828. Min Range: -4.6828

Max Act: 3.9948. Min Act: -0.1700

Data Index: 1636578 (Open Web Text)

Max Activating Token Index: 538

Text #10

Max Range: 4.6828. Min Range: -4.6828

Max Act: 3.9912. Min Act: -0.1700

Data Index: 1232938 (Open Web Text)

Max Activating Token Index: 772

Text #11

Max Range: 4.6828. Min Range: -4.6828

Max Act: 4.0232. Min Act: -0.1700

Data Index: 7159815 (Open Web Text)

Max Activating Token Index: 928

Text #12

Max Range: 4.6828. Min Range: -4.6828

Max Act: 3.9662. Min Act: -0.1700

Data Index: 8065821 (Open Web Text)

Max Activating Token Index: 987

Text #13

Max Range: 4.6828. Min Range: -4.6828

Max Act: 3.9437. Min Act: -0.1700

Data Index: 8337761 (Open Web Text)

Max Activating Token Index: 661

Text #14

Max Range: 4.6828. Min Range: -4.6828

Max Act: 3.9484. Min Act: -0.1700

Data Index: 5540059 (Open Web Text)

Max Activating Token Index: 573

Text #15

Max Range: 4.6828. Min Range: -4.6828

Max Act: 3.8938. Min Act: -0.1700

Data Index: 819121 (Open Web Text)

Max Activating Token Index: 805

Text #16

Max Range: 4.6828. Min Range: -4.6828

Max Act: 3.9345. Min Act: -0.1700

Data Index: 7675871 (Open Web Text)

Max Activating Token Index: 1008

Text #17

Max Range: 4.6828. Min Range: -4.6828

Max Act: 3.9270. Min Act: -0.1700

Data Index: 7148904 (Open Web Text)

Max Activating Token Index: 974

Text #18

Max Range: 4.6828. Min Range: -4.6828

Max Act: 3.9307. Min Act: -0.1700

Data Index: 6612790 (Open Web Text)

Max Activating Token Index: 669

Text #19

Max Range: 4.6828. Min Range: -4.6828

Max Act: 3.9302. Min Act: -0.1700

Data Index: 4835733 (Open Web Text)

Max Activating Token Index: 395