Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2679 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 5.0297. Min Range: -5.0297

Max Act: 5.0297. Min Act: -0.1700

Data Index: 2323901 (Open Web Text)

Max Activating Token Index: 800

Text #1

Max Range: 5.0297. Min Range: -5.0297

Max Act: 4.9321. Min Act: -0.1700

Data Index: 2951432 (Open Web Text)

Max Activating Token Index: 808

Text #2

Max Range: 5.0297. Min Range: -5.0297

Max Act: 4.9515. Min Act: -0.1700

Data Index: 6309444 (Open Web Text)

Max Activating Token Index: 500

Text #3

Max Range: 5.0297. Min Range: -5.0297

Max Act: 4.9524. Min Act: -0.1700

Data Index: 6199449 (Open Web Text)

Max Activating Token Index: 176

Text #4

Max Range: 5.0297. Min Range: -5.0297

Max Act: 4.9305. Min Act: -0.1700

Data Index: 7777499 (Open Web Text)

Max Activating Token Index: 281

Text #5

Max Range: 5.0297. Min Range: -5.0297

Max Act: 4.8910. Min Act: -0.1700

Data Index: 2054952 (Open Web Text)

Max Activating Token Index: 28

Text #6

Max Range: 5.0297. Min Range: -5.0297

Max Act: 4.8695. Min Act: -0.1700

Data Index: 6064253 (Open Web Text)

Max Activating Token Index: 585

Text #7

Max Range: 5.0297. Min Range: -5.0297

Max Act: 4.9046. Min Act: -0.1700

Data Index: 1174575 (Open Web Text)

Max Activating Token Index: 952

Text #8

Max Range: 5.0297. Min Range: -5.0297

Max Act: 4.8537. Min Act: -0.1700

Data Index: 1379479 (Open Web Text)

Max Activating Token Index: 71

Text #9

Max Range: 5.0297. Min Range: -5.0297

Max Act: 4.8249. Min Act: -0.1700

Data Index: 8828933 (Open Web Text)

Max Activating Token Index: 421

Text #10

Max Range: 5.0297. Min Range: -5.0297

Max Act: 4.8572. Min Act: -0.1700

Data Index: 3503042 (Open Web Text)

Max Activating Token Index: 429

Text #11

Max Range: 5.0297. Min Range: -5.0297

Max Act: 4.8161. Min Act: -0.1700

Data Index: 1784671 (Open Web Text)

Max Activating Token Index: 529

Text #12

Max Range: 5.0297. Min Range: -5.0297

Max Act: 4.8294. Min Act: -0.1700

Data Index: 7682602 (Open Web Text)

Max Activating Token Index: 364

Text #13

Max Range: 5.0297. Min Range: -5.0297

Max Act: 4.7998. Min Act: -0.1700

Data Index: 3019907 (Open Web Text)

Max Activating Token Index: 375

Text #14

Max Range: 5.0297. Min Range: -5.0297

Max Act: 4.8433. Min Act: -0.1700

Data Index: 6494741 (Open Web Text)

Max Activating Token Index: 385

Text #15

Max Range: 5.0297. Min Range: -5.0297

Max Act: 4.7753. Min Act: -0.1700

Data Index: 1950071 (Open Web Text)

Max Activating Token Index: 316

Text #16

Max Range: 5.0297. Min Range: -5.0297

Max Act: 4.7848. Min Act: -0.1700

Data Index: 6185206 (Open Web Text)

Max Activating Token Index: 470

Text #17

Max Range: 5.0297. Min Range: -5.0297

Max Act: 4.7879. Min Act: -0.1700

Data Index: 4401854 (Open Web Text)

Max Activating Token Index: 592

Text #18

Max Range: 5.0297. Min Range: -5.0297

Max Act: 4.7849. Min Act: -0.1700

Data Index: 1042399 (Open Web Text)

Max Activating Token Index: 989

Text #19

Max Range: 5.0297. Min Range: -5.0297

Max Act: 4.7700. Min Act: -0.1700

Data Index: 1053166 (Open Web Text)

Max Activating Token Index: 643