Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 441 in Layer 3

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.9640. Min Range: -4.9640

Max Act: 4.9640. Min Act: -0.1700

Data Index: 6496499 (Open Web Text)

Max Activating Token Index: 140

Text #1

Max Range: 4.9640. Min Range: -4.9640

Max Act: 4.9119. Min Act: -0.1700

Data Index: 4364982 (Open Web Text)

Max Activating Token Index: 886

Text #2

Max Range: 4.9640. Min Range: -4.9640

Max Act: 4.8891. Min Act: -0.1700

Data Index: 2622852 (Open Web Text)

Max Activating Token Index: 135

Text #3

Max Range: 4.9640. Min Range: -4.9640

Max Act: 4.8928. Min Act: -0.1700

Data Index: 1422318 (Open Web Text)

Max Activating Token Index: 330

Text #4

Max Range: 4.9640. Min Range: -4.9640

Max Act: 4.8124. Min Act: -0.1700

Data Index: 5211423 (Open Web Text)

Max Activating Token Index: 440

Text #5

Max Range: 4.9640. Min Range: -4.9640

Max Act: 4.7168. Min Act: -0.1700

Data Index: 2511913 (Open Web Text)

Max Activating Token Index: 4

Text #6

Max Range: 4.9640. Min Range: -4.9640

Max Act: 4.6996. Min Act: -0.1700

Data Index: 5984067 (Open Web Text)

Max Activating Token Index: 61

Text #7

Max Range: 4.9640. Min Range: -4.9640

Max Act: 4.6653. Min Act: -0.1700

Data Index: 298816 (Open Web Text)

Max Activating Token Index: 412

Text #8

Max Range: 4.9640. Min Range: -4.9640

Max Act: 4.6951. Min Act: -0.1700

Data Index: 1881159 (Open Web Text)

Max Activating Token Index: 513

Text #9

Max Range: 4.9640. Min Range: -4.9640

Max Act: 4.6942. Min Act: -0.1700

Data Index: 1649877 (Open Web Text)

Max Activating Token Index: 955

Text #10

Max Range: 4.9640. Min Range: -4.9640

Max Act: 4.6846. Min Act: -0.1700

Data Index: 2175368 (Open Web Text)

Max Activating Token Index: 855

Text #11

Max Range: 4.9640. Min Range: -4.9640

Max Act: 4.7030. Min Act: -0.1700

Data Index: 5695183 (Open Web Text)

Max Activating Token Index: 369

Text #12

Max Range: 4.9640. Min Range: -4.9640

Max Act: 4.6994. Min Act: -0.1700

Data Index: 6777362 (Open Web Text)

Max Activating Token Index: 436

Text #13

Max Range: 4.9640. Min Range: -4.9640

Max Act: 4.6555. Min Act: -0.1700

Data Index: 2668823 (Open Web Text)

Max Activating Token Index: 212

Text #14

Max Range: 4.9640. Min Range: -4.9640

Max Act: 4.6585. Min Act: -0.1700

Data Index: 3988083 (Open Web Text)

Max Activating Token Index: 859

Text #15

Max Range: 4.9640. Min Range: -4.9640

Max Act: 4.6601. Min Act: -0.1700

Data Index: 3009685 (Open Web Text)

Max Activating Token Index: 59

Text #16

Max Range: 4.9640. Min Range: -4.9640

Max Act: 4.7166. Min Act: -0.1700

Data Index: 5586564 (Open Web Text)

Max Activating Token Index: 785

Text #17

Max Range: 4.9640. Min Range: -4.9640

Max Act: 4.5759. Min Act: -0.1700

Data Index: 3470170 (Open Web Text)

Max Activating Token Index: 11

Text #18

Max Range: 4.9640. Min Range: -4.9640

Max Act: 4.6407. Min Act: -0.1700

Data Index: 3145085 (Open Web Text)

Max Activating Token Index: 968

Text #19

Max Range: 4.9640. Min Range: -4.9640

Max Act: 4.6298. Min Act: -0.1700

Data Index: 4357133 (Open Web Text)

Max Activating Token Index: 1015