Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 614 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 5.4358. Min Range: -5.4358

Max Act: 5.4358. Min Act: -0.1700

Data Index: 2662980 (Open Web Text)

Max Activating Token Index: 7

Text #1

Max Range: 5.4358. Min Range: -5.4358

Max Act: 5.4039. Min Act: -0.1700

Data Index: 3584412 (Open Web Text)

Max Activating Token Index: 18

Text #2

Max Range: 5.4358. Min Range: -5.4358

Max Act: 5.3790. Min Act: -0.1700

Data Index: 7741255 (Open Web Text)

Max Activating Token Index: 4

Text #3

Max Range: 5.4358. Min Range: -5.4358

Max Act: 5.3514. Min Act: -0.1700

Data Index: 411885 (Open Web Text)

Max Activating Token Index: 5

Text #4

Max Range: 5.4358. Min Range: -5.4358

Max Act: 5.3467. Min Act: -0.1700

Data Index: 5145834 (Open Web Text)

Max Activating Token Index: 6

Text #5

Max Range: 5.4358. Min Range: -5.4358

Max Act: 5.3467. Min Act: -0.1700

Data Index: 8566414 (Open Web Text)

Max Activating Token Index: 6

Text #6

Max Range: 5.4358. Min Range: -5.4358

Max Act: 5.3522. Min Act: -0.1700

Data Index: 781729 (Open Web Text)

Max Activating Token Index: 4

Text #7

Max Range: 5.4358. Min Range: -5.4358

Max Act: 5.3100. Min Act: -0.1700

Data Index: 8744644 (Open Web Text)

Max Activating Token Index: 8

Text #8

Max Range: 5.4358. Min Range: -5.4358

Max Act: 5.3049. Min Act: -0.1700

Data Index: 2315794 (Open Web Text)

Max Activating Token Index: 6

Text #9

Max Range: 5.4358. Min Range: -5.4358

Max Act: 5.3024. Min Act: -0.1700

Data Index: 869984 (Open Web Text)

Max Activating Token Index: 11

Text #10

Max Range: 5.4358. Min Range: -5.4358

Max Act: 5.3276. Min Act: -0.1700

Data Index: 1646568 (Open Web Text)

Max Activating Token Index: 3

Text #11

Max Range: 5.4358. Min Range: -5.4358

Max Act: 5.2837. Min Act: -0.1700

Data Index: 2159691 (Open Web Text)

Max Activating Token Index: 6

Text #12

Max Range: 5.4358. Min Range: -5.4358

Max Act: 5.2780. Min Act: -0.1700

Data Index: 8456790 (Open Web Text)

Max Activating Token Index: 7

Text #13

Max Range: 5.4358. Min Range: -5.4358

Max Act: 5.3079. Min Act: -0.1700

Data Index: 1723232 (Open Web Text)

Max Activating Token Index: 4

Text #14

Max Range: 5.4358. Min Range: -5.4358

Max Act: 5.2941. Min Act: -0.1700

Data Index: 1360373 (Open Web Text)

Max Activating Token Index: 11

Text #15

Max Range: 5.4358. Min Range: -5.4358

Max Act: 5.2948. Min Act: -0.1700

Data Index: 7550816 (Open Web Text)

Max Activating Token Index: 4

Text #16

Max Range: 5.4358. Min Range: -5.4358

Max Act: 5.2694. Min Act: -0.1700

Data Index: 1275958 (Open Web Text)

Max Activating Token Index: 5

Text #17

Max Range: 5.4358. Min Range: -5.4358

Max Act: 5.3079. Min Act: -0.1700

Data Index: 7018901 (Open Web Text)

Max Activating Token Index: 4

Text #18

Max Range: 5.4358. Min Range: -5.4358

Max Act: 5.2763. Min Act: -0.1700

Data Index: 1171127 (Open Web Text)

Max Activating Token Index: 6

Text #19

Max Range: 5.4358. Min Range: -5.4358

Max Act: 5.2858. Min Act: -0.1700

Data Index: 5479403 (Open Web Text)

Max Activating Token Index: 5