Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 818 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.7232. Min Range: -3.7232

Max Act: 3.7232. Min Act: -0.1700

Data Index: 273655 (Open Web Text)

Max Activating Token Index: 989

Text #1

Max Range: 3.7232. Min Range: -3.7232

Max Act: 3.6519. Min Act: -0.1700

Data Index: 3109504 (Open Web Text)

Max Activating Token Index: 849

Text #2

Max Range: 3.7232. Min Range: -3.7232

Max Act: 3.6692. Min Act: -0.1700

Data Index: 5207820 (Open Web Text)

Max Activating Token Index: 949

Text #3

Max Range: 3.7232. Min Range: -3.7232

Max Act: 3.6318. Min Act: -0.1700

Data Index: 5876375 (Open Web Text)

Max Activating Token Index: 555

Text #4

Max Range: 3.7232. Min Range: -3.7232

Max Act: 3.6476. Min Act: -0.1700

Data Index: 8608817 (Open Web Text)

Max Activating Token Index: 854

Text #5

Max Range: 3.7232. Min Range: -3.7232

Max Act: 3.6185. Min Act: -0.1700

Data Index: 3482499 (Open Web Text)

Max Activating Token Index: 251

Text #6

Max Range: 3.7232. Min Range: -3.7232

Max Act: 3.6472. Min Act: -0.1700

Data Index: 819022 (Open Web Text)

Max Activating Token Index: 777

Text #7

Max Range: 3.7232. Min Range: -3.7232

Max Act: 3.6166. Min Act: -0.1700

Data Index: 176680 (Open Web Text)

Max Activating Token Index: 375

Text #8

Max Range: 3.7232. Min Range: -3.7232

Max Act: 3.6012. Min Act: -0.1700

Data Index: 457318 (Open Web Text)

Max Activating Token Index: 254

Text #9

Max Range: 3.7232. Min Range: -3.7232

Max Act: 3.5590. Min Act: -0.1700

Data Index: 535158 (Open Web Text)

Max Activating Token Index: 150

Text #10

Max Range: 3.7232. Min Range: -3.7232

Max Act: 3.5758. Min Act: -0.1700

Data Index: 7583567 (Open Web Text)

Max Activating Token Index: 925

Text #11

Max Range: 3.7232. Min Range: -3.7232

Max Act: 3.5811. Min Act: -0.1700

Data Index: 5205574 (Open Web Text)

Max Activating Token Index: 608

Text #12

Max Range: 3.7232. Min Range: -3.7232

Max Act: 3.5744. Min Act: -0.1700

Data Index: 6411355 (Open Web Text)

Max Activating Token Index: 30

Text #13

Max Range: 3.7232. Min Range: -3.7232

Max Act: 3.5892. Min Act: -0.1700

Data Index: 2761355 (Open Web Text)

Max Activating Token Index: 980

Text #14

Max Range: 3.7232. Min Range: -3.7232

Max Act: 3.5565. Min Act: -0.1700

Data Index: 5045463 (Open Web Text)

Max Activating Token Index: 933

Text #15

Max Range: 3.7232. Min Range: -3.7232

Max Act: 3.5287. Min Act: -0.1700

Data Index: 7363974 (Open Web Text)

Max Activating Token Index: 570

Text #16

Max Range: 3.7232. Min Range: -3.7232

Max Act: 3.5461. Min Act: -0.1700

Data Index: 4526832 (Open Web Text)

Max Activating Token Index: 696

Text #17

Max Range: 3.7232. Min Range: -3.7232

Max Act: 3.5512. Min Act: -0.1700

Data Index: 7428780 (Open Web Text)

Max Activating Token Index: 35

Text #18

Max Range: 3.7232. Min Range: -3.7232

Max Act: 3.5442. Min Act: -0.1700

Data Index: 312250 (Open Web Text)

Max Activating Token Index: 20

Text #19

Max Range: 3.7232. Min Range: -3.7232

Max Act: 3.5267. Min Act: -0.1700

Data Index: 8151739 (Open Web Text)

Max Activating Token Index: 766