Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1080 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.2892. Min Range: -3.2892

Max Act: 3.2892. Min Act: -0.1700

Data Index: 6463534 (Open Web Text)

Max Activating Token Index: 129

Text #1

Max Range: 3.2892. Min Range: -3.2892

Max Act: 3.0228. Min Act: -0.1700

Data Index: 6278157 (Open Web Text)

Max Activating Token Index: 529

Text #2

Max Range: 3.2892. Min Range: -3.2892

Max Act: 2.9988. Min Act: -0.1700

Data Index: 5689 (Open Web Text)

Max Activating Token Index: 304

Text #3

Max Range: 3.2892. Min Range: -3.2892

Max Act: 2.9702. Min Act: -0.1700

Data Index: 1331431 (Open Web Text)

Max Activating Token Index: 165

Text #4

Max Range: 3.2892. Min Range: -3.2892

Max Act: 2.8987. Min Act: -0.1700

Data Index: 4083739 (Open Web Text)

Max Activating Token Index: 535

Text #5

Max Range: 3.2892. Min Range: -3.2892

Max Act: 2.8899. Min Act: -0.1700

Data Index: 1783338 (Open Web Text)

Max Activating Token Index: 494

Text #6

Max Range: 3.2892. Min Range: -3.2892

Max Act: 2.8837. Min Act: -0.1700

Data Index: 2340284 (Open Web Text)

Max Activating Token Index: 180

Text #7

Max Range: 3.2892. Min Range: -3.2892

Max Act: 2.8859. Min Act: -0.1700

Data Index: 2395512 (Open Web Text)

Max Activating Token Index: 681

Text #8

Max Range: 3.2892. Min Range: -3.2892

Max Act: 2.8771. Min Act: -0.1700

Data Index: 1833771 (Open Web Text)

Max Activating Token Index: 432

Text #9

Max Range: 3.2892. Min Range: -3.2892

Max Act: 2.8439. Min Act: -0.1700

Data Index: 7937514 (Open Web Text)

Max Activating Token Index: 789

Text #10

Max Range: 3.2892. Min Range: -3.2892

Max Act: 2.8568. Min Act: -0.1700

Data Index: 4326708 (Open Web Text)

Max Activating Token Index: 131

Text #11

Max Range: 3.2892. Min Range: -3.2892

Max Act: 2.8328. Min Act: -0.1700

Data Index: 4342256 (Open Web Text)

Max Activating Token Index: 28

Text #12

Max Range: 3.2892. Min Range: -3.2892

Max Act: 2.8475. Min Act: -0.1700

Data Index: 5239517 (Open Web Text)

Max Activating Token Index: 358

Text #13

Max Range: 3.2892. Min Range: -3.2892

Max Act: 2.8131. Min Act: -0.1700

Data Index: 7213881 (Open Web Text)

Max Activating Token Index: 749

Text #14

Max Range: 3.2892. Min Range: -3.2892

Max Act: 2.8327. Min Act: -0.1700

Data Index: 961868 (Open Web Text)

Max Activating Token Index: 334

Text #15

Max Range: 3.2892. Min Range: -3.2892

Max Act: 2.8145. Min Act: -0.1700

Data Index: 755723 (Open Web Text)

Max Activating Token Index: 56

Text #16

Max Range: 3.2892. Min Range: -3.2892

Max Act: 2.8198. Min Act: -0.1700

Data Index: 1534209 (Open Web Text)

Max Activating Token Index: 323

Text #17

Max Range: 3.2892. Min Range: -3.2892

Max Act: 2.8282. Min Act: -0.1700

Data Index: 1158095 (Open Web Text)

Max Activating Token Index: 196

Text #18

Max Range: 3.2892. Min Range: -3.2892

Max Act: 2.8230. Min Act: -0.1700

Data Index: 516016 (Open Web Text)

Max Activating Token Index: 103

Text #19

Max Range: 3.2892. Min Range: -3.2892

Max Act: 2.7948. Min Act: -0.1700

Data Index: 4217801 (Open Web Text)

Max Activating Token Index: 588