Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 433 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.3385. Min Range: -4.3385

Max Act: 4.3385. Min Act: -0.1700

Data Index: 1038720 (Open Web Text)

Max Activating Token Index: 511

Text #1

Max Range: 4.3385. Min Range: -4.3385

Max Act: 4.3132. Min Act: -0.1700

Data Index: 3394442 (Open Web Text)

Max Activating Token Index: 272

Text #2

Max Range: 4.3385. Min Range: -4.3385

Max Act: 4.1882. Min Act: -0.1700

Data Index: 2984430 (Open Web Text)

Max Activating Token Index: 193

Text #3

Max Range: 4.3385. Min Range: -4.3385

Max Act: 4.1194. Min Act: -0.1700

Data Index: 546415 (Open Web Text)

Max Activating Token Index: 675

Text #4

Max Range: 4.3385. Min Range: -4.3385

Max Act: 4.1559. Min Act: -0.1700

Data Index: 2307136 (Open Web Text)

Max Activating Token Index: 466

Text #5

Max Range: 4.3385. Min Range: -4.3385

Max Act: 4.1155. Min Act: -0.1700

Data Index: 8382459 (Open Web Text)

Max Activating Token Index: 275

Text #6

Max Range: 4.3385. Min Range: -4.3385

Max Act: 4.0588. Min Act: -0.1700

Data Index: 6861005 (Open Web Text)

Max Activating Token Index: 966

Text #7

Max Range: 4.3385. Min Range: -4.3385

Max Act: 4.0419. Min Act: -0.1700

Data Index: 559851 (Open Web Text)

Max Activating Token Index: 139

Text #8

Max Range: 4.3385. Min Range: -4.3385

Max Act: 4.0628. Min Act: -0.1700

Data Index: 2631355 (Open Web Text)

Max Activating Token Index: 295

Text #9

Max Range: 4.3385. Min Range: -4.3385

Max Act: 4.0391. Min Act: -0.1700

Data Index: 3431375 (Open Web Text)

Max Activating Token Index: 37

Text #10

Max Range: 4.3385. Min Range: -4.3385

Max Act: 3.9955. Min Act: -0.1700

Data Index: 7351390 (Open Web Text)

Max Activating Token Index: 964

Text #11

Max Range: 4.3385. Min Range: -4.3385

Max Act: 4.0053. Min Act: -0.1700

Data Index: 2265761 (Open Web Text)

Max Activating Token Index: 371

Text #12

Max Range: 4.3385. Min Range: -4.3385

Max Act: 3.9897. Min Act: -0.1700

Data Index: 5851051 (Open Web Text)

Max Activating Token Index: 844

Text #13

Max Range: 4.3385. Min Range: -4.3385

Max Act: 3.9981. Min Act: -0.1700

Data Index: 470893 (Open Web Text)

Max Activating Token Index: 363

Text #14

Max Range: 4.3385. Min Range: -4.3385

Max Act: 3.9654. Min Act: -0.1700

Data Index: 2271833 (Open Web Text)

Max Activating Token Index: 238

Text #15

Max Range: 4.3385. Min Range: -4.3385

Max Act: 3.9860. Min Act: -0.1700

Data Index: 3419137 (Open Web Text)

Max Activating Token Index: 113

Text #16

Max Range: 4.3385. Min Range: -4.3385

Max Act: 3.9998. Min Act: -0.1700

Data Index: 7533155 (Open Web Text)

Max Activating Token Index: 947

Text #17

Max Range: 4.3385. Min Range: -4.3385

Max Act: 3.9639. Min Act: -0.1700

Data Index: 4117761 (Open Web Text)

Max Activating Token Index: 737

Text #18

Max Range: 4.3385. Min Range: -4.3385

Max Act: 3.9780. Min Act: -0.1700

Data Index: 4234403 (Open Web Text)

Max Activating Token Index: 959

Text #19

Max Range: 4.3385. Min Range: -4.3385

Max Act: 3.9865. Min Act: -0.1700

Data Index: 6770711 (Open Web Text)

Max Activating Token Index: 170