Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 354 in Layer 4

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 6.5003. Min Range: -6.5003

Max Act: 6.5003. Min Act: -0.1700

Data Index: 6565083 (Open Web Text)

Max Activating Token Index: 98

Text #1

Max Range: 6.5003. Min Range: -6.5003

Max Act: 6.4511. Min Act: -0.1700

Data Index: 565814 (Open Web Text)

Max Activating Token Index: 407

Text #2

Max Range: 6.5003. Min Range: -6.5003

Max Act: 6.2992. Min Act: -0.1700

Data Index: 416638 (Open Web Text)

Max Activating Token Index: 1015

Text #3

Max Range: 6.5003. Min Range: -6.5003

Max Act: 6.2397. Min Act: -0.1700

Data Index: 6226624 (Open Web Text)

Max Activating Token Index: 886

Text #4

Max Range: 6.5003. Min Range: -6.5003

Max Act: 6.2140. Min Act: -0.1700

Data Index: 3821505 (Open Web Text)

Max Activating Token Index: 851

Text #5

Max Range: 6.5003. Min Range: -6.5003

Max Act: 6.1026. Min Act: -0.1700

Data Index: 7305322 (Open Web Text)

Max Activating Token Index: 360

Text #6

Max Range: 6.5003. Min Range: -6.5003

Max Act: 5.9859. Min Act: -0.1700

Data Index: 5873190 (Open Web Text)

Max Activating Token Index: 885

Text #7

Max Range: 6.5003. Min Range: -6.5003

Max Act: 6.0102. Min Act: -0.1700

Data Index: 2278044 (Open Web Text)

Max Activating Token Index: 671

Text #8

Max Range: 6.5003. Min Range: -6.5003

Max Act: 5.9607. Min Act: -0.1700

Data Index: 6077771 (Open Web Text)

Max Activating Token Index: 472

Text #9

Max Range: 6.5003. Min Range: -6.5003

Max Act: 5.9955. Min Act: -0.1700

Data Index: 3325089 (Open Web Text)

Max Activating Token Index: 196

Text #10

Max Range: 6.5003. Min Range: -6.5003

Max Act: 5.8851. Min Act: -0.1700

Data Index: 8135460 (Open Web Text)

Max Activating Token Index: 962

Text #11

Max Range: 6.5003. Min Range: -6.5003

Max Act: 5.8834. Min Act: -0.1700

Data Index: 6365649 (Open Web Text)

Max Activating Token Index: 201

Text #12

Max Range: 6.5003. Min Range: -6.5003

Max Act: 5.8879. Min Act: -0.1700

Data Index: 4192062 (Open Web Text)

Max Activating Token Index: 44

Text #13

Max Range: 6.5003. Min Range: -6.5003

Max Act: 5.9278. Min Act: -0.1700

Data Index: 4552164 (Open Web Text)

Max Activating Token Index: 531

Text #14

Max Range: 6.5003. Min Range: -6.5003

Max Act: 5.8636. Min Act: -0.1700

Data Index: 5910987 (Open Web Text)

Max Activating Token Index: 560

Text #15

Max Range: 6.5003. Min Range: -6.5003

Max Act: 5.8963. Min Act: -0.1700

Data Index: 2153121 (Open Web Text)

Max Activating Token Index: 186

Text #16

Max Range: 6.5003. Min Range: -6.5003

Max Act: 5.8200. Min Act: -0.1700

Data Index: 5643320 (Open Web Text)

Max Activating Token Index: 1003

Text #17

Max Range: 6.5003. Min Range: -6.5003

Max Act: 5.8051. Min Act: -0.1700

Data Index: 2443240 (Open Web Text)

Max Activating Token Index: 940

Text #18

Max Range: 6.5003. Min Range: -6.5003

Max Act: 5.8372. Min Act: -0.1700

Data Index: 6425648 (Open Web Text)

Max Activating Token Index: 481

Text #19

Max Range: 6.5003. Min Range: -6.5003

Max Act: 5.8475. Min Act: -0.1700

Data Index: 3074682 (Open Web Text)

Max Activating Token Index: 410