Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 989 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 5.6320. Min Range: -5.6320

Max Act: 5.6320. Min Act: -0.1700

Data Index: 5720744 (Open Web Text)

Max Activating Token Index: 9

Text #1

Max Range: 5.6320. Min Range: -5.6320

Max Act: 5.6055. Min Act: -0.1700

Data Index: 8110329 (Open Web Text)

Max Activating Token Index: 15

Text #2

Max Range: 5.6320. Min Range: -5.6320

Max Act: 5.5373. Min Act: -0.1700

Data Index: 5293535 (Open Web Text)

Max Activating Token Index: 10

Text #3

Max Range: 5.6320. Min Range: -5.6320

Max Act: 5.5663. Min Act: -0.1700

Data Index: 1902265 (Open Web Text)

Max Activating Token Index: 10

Text #4

Max Range: 5.6320. Min Range: -5.6320

Max Act: 5.5277. Min Act: -0.1700

Data Index: 1910886 (Open Web Text)

Max Activating Token Index: 8

Text #5

Max Range: 5.6320. Min Range: -5.6320

Max Act: 5.5447. Min Act: -0.1700

Data Index: 3001099 (Open Web Text)

Max Activating Token Index: 6

Text #6

Max Range: 5.6320. Min Range: -5.6320

Max Act: 5.4844. Min Act: -0.1700

Data Index: 5888856 (Open Web Text)

Max Activating Token Index: 8

Text #7

Max Range: 5.6320. Min Range: -5.6320

Max Act: 5.4865. Min Act: -0.1700

Data Index: 7561133 (Open Web Text)

Max Activating Token Index: 9

Text #8

Max Range: 5.6320. Min Range: -5.6320

Max Act: 5.4689. Min Act: -0.1700

Data Index: 954742 (Open Web Text)

Max Activating Token Index: 20

Text #9

Max Range: 5.6320. Min Range: -5.6320

Max Act: 5.4628. Min Act: -0.1700

Data Index: 820326 (Open Web Text)

Max Activating Token Index: 9

Text #10

Max Range: 5.6320. Min Range: -5.6320

Max Act: 5.4409. Min Act: -0.1700

Data Index: 828204 (Open Web Text)

Max Activating Token Index: 9

Text #11

Max Range: 5.6320. Min Range: -5.6320

Max Act: 5.4250. Min Act: -0.1700

Data Index: 7031460 (Open Web Text)

Max Activating Token Index: 15

Text #12

Max Range: 5.6320. Min Range: -5.6320

Max Act: 5.4254. Min Act: -0.1700

Data Index: 7220876 (Open Web Text)

Max Activating Token Index: 6

Text #13

Max Range: 5.6320. Min Range: -5.6320

Max Act: 5.4251. Min Act: -0.1700

Data Index: 3877824 (Open Web Text)

Max Activating Token Index: 10

Text #14

Max Range: 5.6320. Min Range: -5.6320

Max Act: 5.4253. Min Act: -0.1700

Data Index: 2920912 (Open Web Text)

Max Activating Token Index: 8

Text #15

Max Range: 5.6320. Min Range: -5.6320

Max Act: 5.4288. Min Act: -0.1700

Data Index: 2798114 (Open Web Text)

Max Activating Token Index: 12

Text #16

Max Range: 5.6320. Min Range: -5.6320

Max Act: 5.4202. Min Act: -0.1700

Data Index: 6955962 (Open Web Text)

Max Activating Token Index: 9

Text #17

Max Range: 5.6320. Min Range: -5.6320

Max Act: 5.4444. Min Act: -0.1700

Data Index: 4714628 (Open Web Text)

Max Activating Token Index: 14

Text #18

Max Range: 5.6320. Min Range: -5.6320

Max Act: 5.4504. Min Act: -0.1700

Data Index: 5172815 (Open Web Text)

Max Activating Token Index: 7

Text #19

Max Range: 5.6320. Min Range: -5.6320

Max Act: 5.4289. Min Act: -0.1700

Data Index: 6429151 (Open Web Text)

Max Activating Token Index: 5