Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 400 in Layer 1

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.6585. Min Range: -4.6585

Max Act: 4.6585. Min Act: -0.1700

Data Index: 2367427 (Open Web Text)

Max Activating Token Index: 8

Text #1

Max Range: 4.6585. Min Range: -4.6585

Max Act: 4.6243. Min Act: -0.1700

Data Index: 4147245 (Open Web Text)

Max Activating Token Index: 8

Text #2

Max Range: 4.6585. Min Range: -4.6585

Max Act: 4.5679. Min Act: -0.1700

Data Index: 5620143 (Open Web Text)

Max Activating Token Index: 340

Text #3

Max Range: 4.6585. Min Range: -4.6585

Max Act: 4.5508. Min Act: -0.1700

Data Index: 6854113 (Open Web Text)

Max Activating Token Index: 217

Text #4

Max Range: 4.6585. Min Range: -4.6585

Max Act: 4.5581. Min Act: -0.1700

Data Index: 7249039 (Open Web Text)

Max Activating Token Index: 205

Text #5

Max Range: 4.6585. Min Range: -4.6585

Max Act: 4.5493. Min Act: -0.1700

Data Index: 5736666 (Open Web Text)

Max Activating Token Index: 768

Text #6

Max Range: 4.6585. Min Range: -4.6585

Max Act: 4.5336. Min Act: -0.1700

Data Index: 2562335 (Open Web Text)

Max Activating Token Index: 9

Text #7

Max Range: 4.6585. Min Range: -4.6585

Max Act: 4.5031. Min Act: -0.1700

Data Index: 1851946 (Open Web Text)

Max Activating Token Index: 329

Text #8

Max Range: 4.6585. Min Range: -4.6585

Max Act: 4.5239. Min Act: -0.1700

Data Index: 5191528 (Open Web Text)

Max Activating Token Index: 421

Text #9

Max Range: 4.6585. Min Range: -4.6585

Max Act: 4.5145. Min Act: -0.1700

Data Index: 4975442 (Open Web Text)

Max Activating Token Index: 580

Text #10

Max Range: 4.6585. Min Range: -4.6585

Max Act: 4.5194. Min Act: -0.1700

Data Index: 1444364 (Open Web Text)

Max Activating Token Index: 36

Text #11

Max Range: 4.6585. Min Range: -4.6585

Max Act: 4.5213. Min Act: -0.1698

Data Index: 8089659 (Open Web Text)

Max Activating Token Index: 49

Text #12

Max Range: 4.6585. Min Range: -4.6585

Max Act: 4.5276. Min Act: -0.1700

Data Index: 4064305 (Open Web Text)

Max Activating Token Index: 318

Text #13

Max Range: 4.6585. Min Range: -4.6585

Max Act: 4.5385. Min Act: -0.1700

Data Index: 4000242 (Open Web Text)

Max Activating Token Index: 625

Text #14

Max Range: 4.6585. Min Range: -4.6585

Max Act: 4.4957. Min Act: -0.1700

Data Index: 5053602 (Open Web Text)

Max Activating Token Index: 205

Text #15

Max Range: 4.6585. Min Range: -4.6585

Max Act: 4.5009. Min Act: -0.1700

Data Index: 4294143 (Open Web Text)

Max Activating Token Index: 318

Text #16

Max Range: 4.6585. Min Range: -4.6585

Max Act: 4.4985. Min Act: -0.1700

Data Index: 140181 (Open Web Text)

Max Activating Token Index: 327

Text #17

Max Range: 4.6585. Min Range: -4.6585

Max Act: 4.4870. Min Act: -0.1700

Data Index: 6169359 (Open Web Text)

Max Activating Token Index: 10

Text #18

Max Range: 4.6585. Min Range: -4.6585

Max Act: 4.4983. Min Act: -0.1700

Data Index: 6171930 (Open Web Text)

Max Activating Token Index: 482

Text #19

Max Range: 4.6585. Min Range: -4.6585

Max Act: 4.4855. Min Act: -0.1700

Data Index: 6568868 (Open Web Text)

Max Activating Token Index: 9