Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1485 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 7.3689. Min Range: -7.3689

Max Act: 7.3689. Min Act: -0.1700

Data Index: 3787901 (Open Web Text)

Max Activating Token Index: 395

Text #1

Max Range: 7.3689. Min Range: -7.3689

Max Act: 7.3438. Min Act: -0.1700

Data Index: 190547 (Open Web Text)

Max Activating Token Index: 109

Text #2

Max Range: 7.3689. Min Range: -7.3689

Max Act: 7.2454. Min Act: -0.1700

Data Index: 5075215 (Open Web Text)

Max Activating Token Index: 783

Text #3

Max Range: 7.3689. Min Range: -7.3689

Max Act: 7.2219. Min Act: -0.1700

Data Index: 2094057 (Open Web Text)

Max Activating Token Index: 149

Text #4

Max Range: 7.3689. Min Range: -7.3689

Max Act: 7.1542. Min Act: -0.1700

Data Index: 5639603 (Open Web Text)

Max Activating Token Index: 186

Text #5

Max Range: 7.3689. Min Range: -7.3689

Max Act: 7.1313. Min Act: -0.1700

Data Index: 2168929 (Open Web Text)

Max Activating Token Index: 216

Text #6

Max Range: 7.3689. Min Range: -7.3689

Max Act: 7.1206. Min Act: -0.1700

Data Index: 5753651 (Open Web Text)

Max Activating Token Index: 110

Text #7

Max Range: 7.3689. Min Range: -7.3689

Max Act: 7.1064. Min Act: -0.1700

Data Index: 8079733 (Open Web Text)

Max Activating Token Index: 654

Text #8

Max Range: 7.3689. Min Range: -7.3689

Max Act: 7.1554. Min Act: -0.1700

Data Index: 3474566 (Open Web Text)

Max Activating Token Index: 477

Text #9

Max Range: 7.3689. Min Range: -7.3689

Max Act: 7.1278. Min Act: -0.1700

Data Index: 4646476 (Open Web Text)

Max Activating Token Index: 431

Text #10

Max Range: 7.3689. Min Range: -7.3689

Max Act: 7.1247. Min Act: -0.1700

Data Index: 1856341 (Open Web Text)

Max Activating Token Index: 1003

Text #11

Max Range: 7.3689. Min Range: -7.3689

Max Act: 7.0991. Min Act: -0.1700

Data Index: 5653339 (Open Web Text)

Max Activating Token Index: 869

Text #12

Max Range: 7.3689. Min Range: -7.3689

Max Act: 7.1205. Min Act: -0.1700

Data Index: 1985793 (Open Web Text)

Max Activating Token Index: 984

Text #13

Max Range: 7.3689. Min Range: -7.3689

Max Act: 7.0876. Min Act: -0.1700

Data Index: 2347880 (Open Web Text)

Max Activating Token Index: 984

Text #14

Max Range: 7.3689. Min Range: -7.3689

Max Act: 7.0783. Min Act: -0.1700

Data Index: 5937735 (Open Web Text)

Max Activating Token Index: 144

Text #15

Max Range: 7.3689. Min Range: -7.3689

Max Act: 7.1007. Min Act: -0.1700

Data Index: 1535554 (Open Web Text)

Max Activating Token Index: 289

Text #16

Max Range: 7.3689. Min Range: -7.3689

Max Act: 6.9927. Min Act: -0.1700

Data Index: 8180844 (Open Web Text)

Max Activating Token Index: 515

Text #17

Max Range: 7.3689. Min Range: -7.3689

Max Act: 7.0085. Min Act: -0.1700

Data Index: 5691798 (Open Web Text)

Max Activating Token Index: 59

Text #18

Max Range: 7.3689. Min Range: -7.3689

Max Act: 6.9659. Min Act: -0.1700

Data Index: 5720635 (Open Web Text)

Max Activating Token Index: 788

Text #19

Max Range: 7.3689. Min Range: -7.3689

Max Act: 6.9164. Min Act: -0.1700

Data Index: 211518 (Open Web Text)

Max Activating Token Index: 585