Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 223 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.3010. Min Range: -4.3010

Max Act: 4.3010. Min Act: -0.1700

Data Index: 8273675 (Open Web Text)

Max Activating Token Index: 577

Text #1

Max Range: 4.3010. Min Range: -4.3010

Max Act: 4.2623. Min Act: -0.1700

Data Index: 2974636 (Open Web Text)

Max Activating Token Index: 772

Text #2

Max Range: 4.3010. Min Range: -4.3010

Max Act: 4.2572. Min Act: -0.1700

Data Index: 4280619 (Open Web Text)

Max Activating Token Index: 357

Text #3

Max Range: 4.3010. Min Range: -4.3010

Max Act: 4.1956. Min Act: -0.1700

Data Index: 2955210 (Open Web Text)

Max Activating Token Index: 83

Text #4

Max Range: 4.3010. Min Range: -4.3010

Max Act: 4.1530. Min Act: -0.1700

Data Index: 1183397 (Open Web Text)

Max Activating Token Index: 730

Text #5

Max Range: 4.3010. Min Range: -4.3010

Max Act: 4.1294. Min Act: -0.1700

Data Index: 4433881 (Open Web Text)

Max Activating Token Index: 261

Text #6

Max Range: 4.3010. Min Range: -4.3010

Max Act: 4.1308. Min Act: -0.1700

Data Index: 8101380 (Open Web Text)

Max Activating Token Index: 567

Text #7

Max Range: 4.3010. Min Range: -4.3010

Max Act: 4.0823. Min Act: -0.1700

Data Index: 7507555 (Open Web Text)

Max Activating Token Index: 716

Text #8

Max Range: 4.3010. Min Range: -4.3010

Max Act: 4.1113. Min Act: -0.1700

Data Index: 8049193 (Open Web Text)

Max Activating Token Index: 452

Text #9

Max Range: 4.3010. Min Range: -4.3010

Max Act: 4.0679. Min Act: -0.1700

Data Index: 65387 (Open Web Text)

Max Activating Token Index: 966

Text #10

Max Range: 4.3010. Min Range: -4.3010

Max Act: 4.0543. Min Act: -0.1700

Data Index: 6788655 (Open Web Text)

Max Activating Token Index: 651

Text #11

Max Range: 4.3010. Min Range: -4.3010

Max Act: 4.0560. Min Act: -0.1700

Data Index: 8536530 (Open Web Text)

Max Activating Token Index: 787

Text #12

Max Range: 4.3010. Min Range: -4.3010

Max Act: 4.0968. Min Act: -0.1700

Data Index: 3415253 (Open Web Text)

Max Activating Token Index: 955

Text #13

Max Range: 4.3010. Min Range: -4.3010

Max Act: 4.0835. Min Act: -0.1700

Data Index: 7029137 (Open Web Text)

Max Activating Token Index: 470

Text #14

Max Range: 4.3010. Min Range: -4.3010

Max Act: 4.0286. Min Act: -0.1700

Data Index: 7808645 (Open Web Text)

Max Activating Token Index: 139

Text #15

Max Range: 4.3010. Min Range: -4.3010

Max Act: 4.0125. Min Act: -0.1700

Data Index: 7522581 (Open Web Text)

Max Activating Token Index: 363

Text #16

Max Range: 4.3010. Min Range: -4.3010

Max Act: 4.0167. Min Act: -0.1700

Data Index: 6715364 (Open Web Text)

Max Activating Token Index: 340

Text #17

Max Range: 4.3010. Min Range: -4.3010

Max Act: 4.0436. Min Act: -0.1700

Data Index: 2274291 (Open Web Text)

Max Activating Token Index: 566

Text #18

Max Range: 4.3010. Min Range: -4.3010

Max Act: 4.0460. Min Act: -0.1700

Data Index: 1116227 (Open Web Text)

Max Activating Token Index: 244

Text #19

Max Range: 4.3010. Min Range: -4.3010

Max Act: 4.0144. Min Act: -0.1700

Data Index: 3306467 (Open Web Text)

Max Activating Token Index: 318