Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1926 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 1.8548. Min Range: -1.8548

Max Act: 1.8548. Min Act: -0.1638

Data Index: 6346970 (Open Web Text)

Max Activating Token Index: 4

Text #1

Max Range: 1.8548. Min Range: -1.8548

Max Act: 1.8199. Min Act: -0.1700

Data Index: 7541727 (Open Web Text)

Max Activating Token Index: 7

Text #2

Max Range: 1.8548. Min Range: -1.8548

Max Act: 1.7824. Min Act: -0.1700

Data Index: 332587 (Open Web Text)

Max Activating Token Index: 4

Text #3

Max Range: 1.8548. Min Range: -1.8548

Max Act: 1.7833. Min Act: -0.1700

Data Index: 3440036 (Open Web Text)

Max Activating Token Index: 4

Text #4

Max Range: 1.8548. Min Range: -1.8548

Max Act: 1.7770. Min Act: -0.1648

Data Index: 89865 (Open Web Text)

Max Activating Token Index: 2

Text #5

Max Range: 1.8548. Min Range: -1.8548

Max Act: 1.7770. Min Act: -0.1700

Data Index: 929478 (Open Web Text)

Max Activating Token Index: 2

Text #6

Max Range: 1.8548. Min Range: -1.8548

Max Act: 1.7770. Min Act: -0.1700

Data Index: 4471132 (Open Web Text)

Max Activating Token Index: 2

Text #7

Max Range: 1.8548. Min Range: -1.8548

Max Act: 1.7770. Min Act: -0.1579

Data Index: 1367352 (Open Web Text)

Max Activating Token Index: 2

Text #8

Max Range: 1.8548. Min Range: -1.8548

Max Act: 1.7770. Min Act: -0.1700

Data Index: 7360430 (Open Web Text)

Max Activating Token Index: 2

Text #9

Max Range: 1.8548. Min Range: -1.8548

Max Act: 1.7770. Min Act: -0.1700

Data Index: 5447957 (Open Web Text)

Max Activating Token Index: 2

Text #10

Max Range: 1.8548. Min Range: -1.8548

Max Act: 1.7782. Min Act: -0.1700

Data Index: 992000 (Open Web Text)

Max Activating Token Index: 4

Text #11

Max Range: 1.8548. Min Range: -1.8548

Max Act: 1.7504. Min Act: -0.1654

Data Index: 5442986 (Open Web Text)

Max Activating Token Index: 3

Text #12

Max Range: 1.8548. Min Range: -1.8548

Max Act: 1.7519. Min Act: -0.1679

Data Index: 1714719 (Open Web Text)

Max Activating Token Index: 3

Text #13

Max Range: 1.8548. Min Range: -1.8548

Max Act: 1.7346. Min Act: -0.1687

Data Index: 5774969 (Open Web Text)

Max Activating Token Index: 2

Text #14

Max Range: 1.8548. Min Range: -1.8548

Max Act: 1.7350. Min Act: -0.1700

Data Index: 3538094 (Open Web Text)

Max Activating Token Index: 5

Text #15

Max Range: 1.8548. Min Range: -1.8548

Max Act: 1.7346. Min Act: -0.1672

Data Index: 5214933 (Open Web Text)

Max Activating Token Index: 2

Text #16

Max Range: 1.8548. Min Range: -1.8548

Max Act: 1.7346. Min Act: -0.1693

Data Index: 3973208 (Open Web Text)

Max Activating Token Index: 2

Text #17

Max Range: 1.8548. Min Range: -1.8548

Max Act: 1.7319. Min Act: -0.1646

Data Index: 3357934 (Open Web Text)

Max Activating Token Index: 10

Text #18

Max Range: 1.8548. Min Range: -1.8548

Max Act: 1.7346. Min Act: -0.1700

Data Index: 4518968 (Open Web Text)

Max Activating Token Index: 2

Text #19

Max Range: 1.8548. Min Range: -1.8548

Max Act: 1.7272. Min Act: -0.1668

Data Index: 4415043 (Open Web Text)

Max Activating Token Index: 4