Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 403 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.7229. Min Range: -4.7229

Max Act: 4.7229. Min Act: -0.1700

Data Index: 6735525 (Open Web Text)

Max Activating Token Index: 1021

Text #1

Max Range: 4.7229. Min Range: -4.7229

Max Act: 4.6450. Min Act: -0.1700

Data Index: 3734229 (Open Web Text)

Max Activating Token Index: 77

Text #2

Max Range: 4.7229. Min Range: -4.7229

Max Act: 4.6532. Min Act: -0.1700

Data Index: 3608978 (Open Web Text)

Max Activating Token Index: 166

Text #3

Max Range: 4.7229. Min Range: -4.7229

Max Act: 4.6675. Min Act: -0.1700

Data Index: 4034229 (Open Web Text)

Max Activating Token Index: 587

Text #4

Max Range: 4.7229. Min Range: -4.7229

Max Act: 4.6459. Min Act: -0.1700

Data Index: 8609679 (Open Web Text)

Max Activating Token Index: 932

Text #5

Max Range: 4.7229. Min Range: -4.7229

Max Act: 4.6268. Min Act: -0.1700

Data Index: 5287601 (Open Web Text)

Max Activating Token Index: 138

Text #6

Max Range: 4.7229. Min Range: -4.7229

Max Act: 4.6260. Min Act: -0.1700

Data Index: 6428200 (Open Web Text)

Max Activating Token Index: 471

Text #7

Max Range: 4.7229. Min Range: -4.7229

Max Act: 4.5955. Min Act: -0.1700

Data Index: 7640157 (Open Web Text)

Max Activating Token Index: 781

Text #8

Max Range: 4.7229. Min Range: -4.7229

Max Act: 4.6112. Min Act: -0.1700

Data Index: 2280678 (Open Web Text)

Max Activating Token Index: 151

Text #9

Max Range: 4.7229. Min Range: -4.7229

Max Act: 4.5772. Min Act: -0.1700

Data Index: 2909644 (Open Web Text)

Max Activating Token Index: 955

Text #10

Max Range: 4.7229. Min Range: -4.7229

Max Act: 4.5848. Min Act: -0.1700

Data Index: 8738339 (Open Web Text)

Max Activating Token Index: 398

Text #11

Max Range: 4.7229. Min Range: -4.7229

Max Act: 4.6045. Min Act: -0.1700

Data Index: 6475434 (Open Web Text)

Max Activating Token Index: 813

Text #12

Max Range: 4.7229. Min Range: -4.7229

Max Act: 4.5824. Min Act: -0.1700

Data Index: 422270 (Open Web Text)

Max Activating Token Index: 916

Text #13

Max Range: 4.7229. Min Range: -4.7229

Max Act: 4.5639. Min Act: -0.1700

Data Index: 1649824 (Open Web Text)

Max Activating Token Index: 979

Text #14

Max Range: 4.7229. Min Range: -4.7229

Max Act: 4.5688. Min Act: -0.1700

Data Index: 3755039 (Open Web Text)

Max Activating Token Index: 92

Text #15

Max Range: 4.7229. Min Range: -4.7229

Max Act: 4.5754. Min Act: -0.1700

Data Index: 5543777 (Open Web Text)

Max Activating Token Index: 706

Text #16

Max Range: 4.7229. Min Range: -4.7229

Max Act: 4.5571. Min Act: -0.1700

Data Index: 7602560 (Open Web Text)

Max Activating Token Index: 344

Text #17

Max Range: 4.7229. Min Range: -4.7229

Max Act: 4.5560. Min Act: -0.1700

Data Index: 2614792 (Open Web Text)

Max Activating Token Index: 587

Text #18

Max Range: 4.7229. Min Range: -4.7229

Max Act: 4.5456. Min Act: -0.1700

Data Index: 7081805 (Open Web Text)

Max Activating Token Index: 362

Text #19

Max Range: 4.7229. Min Range: -4.7229

Max Act: 4.5169. Min Act: -0.1700

Data Index: 4999478 (Open Web Text)

Max Activating Token Index: 438