Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1678 in Layer 3

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.5578. Min Range: -3.5578

Max Act: 3.5578. Min Act: -0.1700

Data Index: 3440583 (Open Web Text)

Max Activating Token Index: 740

Text #1

Max Range: 3.5578. Min Range: -3.5578

Max Act: 3.5177. Min Act: -0.1700

Data Index: 5999414 (Open Web Text)

Max Activating Token Index: 875

Text #2

Max Range: 3.5578. Min Range: -3.5578

Max Act: 3.5190. Min Act: -0.1700

Data Index: 2224043 (Open Web Text)

Max Activating Token Index: 768

Text #3

Max Range: 3.5578. Min Range: -3.5578

Max Act: 3.4681. Min Act: -0.1700

Data Index: 5437241 (Open Web Text)

Max Activating Token Index: 633

Text #4

Max Range: 3.5578. Min Range: -3.5578

Max Act: 3.4834. Min Act: -0.1700

Data Index: 3021169 (Open Web Text)

Max Activating Token Index: 611

Text #5

Max Range: 3.5578. Min Range: -3.5578

Max Act: 3.4197. Min Act: -0.1700

Data Index: 5498723 (Open Web Text)

Max Activating Token Index: 873

Text #6

Max Range: 3.5578. Min Range: -3.5578

Max Act: 3.4698. Min Act: -0.1700

Data Index: 5565809 (Open Web Text)

Max Activating Token Index: 331

Text #7

Max Range: 3.5578. Min Range: -3.5578

Max Act: 3.4328. Min Act: -0.1700

Data Index: 8722052 (Open Web Text)

Max Activating Token Index: 395

Text #8

Max Range: 3.5578. Min Range: -3.5578

Max Act: 3.4800. Min Act: -0.1700

Data Index: 6205337 (Open Web Text)

Max Activating Token Index: 487

Text #9

Max Range: 3.5578. Min Range: -3.5578

Max Act: 3.4723. Min Act: -0.1700

Data Index: 1063178 (Open Web Text)

Max Activating Token Index: 221

Text #10

Max Range: 3.5578. Min Range: -3.5578

Max Act: 3.4240. Min Act: -0.1700

Data Index: 26790 (Open Web Text)

Max Activating Token Index: 504

Text #11

Max Range: 3.5578. Min Range: -3.5578

Max Act: 3.3837. Min Act: -0.1700

Data Index: 7606041 (Open Web Text)

Max Activating Token Index: 1023

Text #12

Max Range: 3.5578. Min Range: -3.5578

Max Act: 3.4100. Min Act: -0.1700

Data Index: 840198 (Open Web Text)

Max Activating Token Index: 706

Text #13

Max Range: 3.5578. Min Range: -3.5578

Max Act: 3.3825. Min Act: -0.1700

Data Index: 7009922 (Open Web Text)

Max Activating Token Index: 325

Text #14

Max Range: 3.5578. Min Range: -3.5578

Max Act: 3.3853. Min Act: -0.1700

Data Index: 4494130 (Open Web Text)

Max Activating Token Index: 550

Text #15

Max Range: 3.5578. Min Range: -3.5578

Max Act: 3.4277. Min Act: -0.1700

Data Index: 6550276 (Open Web Text)

Max Activating Token Index: 728

Text #16

Max Range: 3.5578. Min Range: -3.5578

Max Act: 3.4116. Min Act: -0.1700

Data Index: 6205339 (Open Web Text)

Max Activating Token Index: 57

Text #17

Max Range: 3.5578. Min Range: -3.5578

Max Act: 3.4031. Min Act: -0.1700

Data Index: 1253157 (Open Web Text)

Max Activating Token Index: 35

Text #18

Max Range: 3.5578. Min Range: -3.5578

Max Act: 3.4133. Min Act: -0.1700

Data Index: 3126761 (Open Web Text)

Max Activating Token Index: 890

Text #19

Max Range: 3.5578. Min Range: -3.5578

Max Act: 3.3904. Min Act: -0.1700

Data Index: 2154286 (Open Web Text)

Max Activating Token Index: 85