Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2982 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.5250. Min Range: -3.5250

Max Act: 3.5250. Min Act: -0.1700

Data Index: 6411599 (Open Web Text)

Max Activating Token Index: 644

Text #1

Max Range: 3.5250. Min Range: -3.5250

Max Act: 3.5026. Min Act: -0.1700

Data Index: 8533955 (Open Web Text)

Max Activating Token Index: 397

Text #2

Max Range: 3.5250. Min Range: -3.5250

Max Act: 3.4386. Min Act: -0.1700

Data Index: 395910 (Open Web Text)

Max Activating Token Index: 496

Text #3

Max Range: 3.5250. Min Range: -3.5250

Max Act: 3.4521. Min Act: -0.1700

Data Index: 6727390 (Open Web Text)

Max Activating Token Index: 431

Text #4

Max Range: 3.5250. Min Range: -3.5250

Max Act: 3.4334. Min Act: -0.1700

Data Index: 7231099 (Open Web Text)

Max Activating Token Index: 626

Text #5

Max Range: 3.5250. Min Range: -3.5250

Max Act: 3.4012. Min Act: -0.1700

Data Index: 6104256 (Open Web Text)

Max Activating Token Index: 678

Text #6

Max Range: 3.5250. Min Range: -3.5250

Max Act: 3.4208. Min Act: -0.1700

Data Index: 6906703 (Open Web Text)

Max Activating Token Index: 316

Text #7

Max Range: 3.5250. Min Range: -3.5250

Max Act: 3.4178. Min Act: -0.1700

Data Index: 3465811 (Open Web Text)

Max Activating Token Index: 842

Text #8

Max Range: 3.5250. Min Range: -3.5250

Max Act: 3.4019. Min Act: -0.1700

Data Index: 6523167 (Open Web Text)

Max Activating Token Index: 219

Text #9

Max Range: 3.5250. Min Range: -3.5250

Max Act: 3.4209. Min Act: -0.1700

Data Index: 3797837 (Open Web Text)

Max Activating Token Index: 62

Text #10

Max Range: 3.5250. Min Range: -3.5250

Max Act: 3.4039. Min Act: -0.1700

Data Index: 3580135 (Open Web Text)

Max Activating Token Index: 472

Text #11

Max Range: 3.5250. Min Range: -3.5250

Max Act: 3.3842. Min Act: -0.1700

Data Index: 1539797 (Open Web Text)

Max Activating Token Index: 342

Text #12

Max Range: 3.5250. Min Range: -3.5250

Max Act: 3.3932. Min Act: -0.1700

Data Index: 221396 (Open Web Text)

Max Activating Token Index: 804

Text #13

Max Range: 3.5250. Min Range: -3.5250

Max Act: 3.3903. Min Act: -0.1700

Data Index: 5111399 (Open Web Text)

Max Activating Token Index: 1011

Text #14

Max Range: 3.5250. Min Range: -3.5250

Max Act: 3.4030. Min Act: -0.1700

Data Index: 8776726 (Open Web Text)

Max Activating Token Index: 482

Text #15

Max Range: 3.5250. Min Range: -3.5250

Max Act: 3.4131. Min Act: -0.1700

Data Index: 8539979 (Open Web Text)

Max Activating Token Index: 800

Text #16

Max Range: 3.5250. Min Range: -3.5250

Max Act: 3.3887. Min Act: -0.1700

Data Index: 3343951 (Open Web Text)

Max Activating Token Index: 584

Text #17

Max Range: 3.5250. Min Range: -3.5250

Max Act: 3.3727. Min Act: -0.1700

Data Index: 196411 (Open Web Text)

Max Activating Token Index: 906

Text #18

Max Range: 3.5250. Min Range: -3.5250

Max Act: 3.3782. Min Act: -0.1700

Data Index: 6642012 (Open Web Text)

Max Activating Token Index: 449

Text #19

Max Range: 3.5250. Min Range: -3.5250

Max Act: 3.3573. Min Act: -0.1700

Data Index: 3785730 (Open Web Text)

Max Activating Token Index: 920