Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 45 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 6.0313. Min Range: -6.0313

Max Act: 6.0287. Min Act: -0.1700

Data Index: 1208899 (Open Web Text)

Max Activating Token Index: 650

Text #1

Max Range: 6.0313. Min Range: -6.0313

Max Act: 6.0102. Min Act: -0.1700

Data Index: 5955834 (Open Web Text)

Max Activating Token Index: 374

Text #2

Max Range: 6.0313. Min Range: -6.0313

Max Act: 6.0102. Min Act: -0.1700

Data Index: 8484747 (Open Web Text)

Max Activating Token Index: 401

Text #3

Max Range: 6.0313. Min Range: -6.0313

Max Act: 6.0313. Min Act: -0.1700

Data Index: 3377449 (Open Web Text)

Max Activating Token Index: 536

Text #4

Max Range: 6.0313. Min Range: -6.0313

Max Act: 5.9705. Min Act: -0.1700

Data Index: 2101482 (Open Web Text)

Max Activating Token Index: 576

Text #5

Max Range: 6.0313. Min Range: -6.0313

Max Act: 5.9275. Min Act: -0.1700

Data Index: 332635 (Open Web Text)

Max Activating Token Index: 248

Text #6

Max Range: 6.0313. Min Range: -6.0313

Max Act: 5.8658. Min Act: -0.1700

Data Index: 1721900 (Open Web Text)

Max Activating Token Index: 254

Text #7

Max Range: 6.0313. Min Range: -6.0313

Max Act: 5.8784. Min Act: -0.1700

Data Index: 3031427 (Open Web Text)

Max Activating Token Index: 437

Text #8

Max Range: 6.0313. Min Range: -6.0313

Max Act: 5.8747. Min Act: -0.1700

Data Index: 7591014 (Open Web Text)

Max Activating Token Index: 496

Text #9

Max Range: 6.0313. Min Range: -6.0313

Max Act: 5.8825. Min Act: -0.1700

Data Index: 763445 (Open Web Text)

Max Activating Token Index: 593

Text #10

Max Range: 6.0313. Min Range: -6.0313

Max Act: 5.8715. Min Act: -0.1700

Data Index: 4430160 (Open Web Text)

Max Activating Token Index: 458

Text #11

Max Range: 6.0313. Min Range: -6.0313

Max Act: 5.8337. Min Act: -0.1700

Data Index: 994806 (Open Web Text)

Max Activating Token Index: 236

Text #12

Max Range: 6.0313. Min Range: -6.0313

Max Act: 5.8262. Min Act: -0.1700

Data Index: 500362 (Open Web Text)

Max Activating Token Index: 486

Text #13

Max Range: 6.0313. Min Range: -6.0313

Max Act: 5.8554. Min Act: -0.1700

Data Index: 4464484 (Open Web Text)

Max Activating Token Index: 144

Text #14

Max Range: 6.0313. Min Range: -6.0313

Max Act: 5.8120. Min Act: -0.1700

Data Index: 6680038 (Open Web Text)

Max Activating Token Index: 237

Text #15

Max Range: 6.0313. Min Range: -6.0313

Max Act: 5.8132. Min Act: -0.1700

Data Index: 5594309 (Open Web Text)

Max Activating Token Index: 42

Text #16

Max Range: 6.0313. Min Range: -6.0313

Max Act: 5.8313. Min Act: -0.1700

Data Index: 5406571 (Open Web Text)

Max Activating Token Index: 332

Text #17

Max Range: 6.0313. Min Range: -6.0313

Max Act: 5.8278. Min Act: -0.1700

Data Index: 7547136 (Open Web Text)

Max Activating Token Index: 949

Text #18

Max Range: 6.0313. Min Range: -6.0313

Max Act: 5.8388. Min Act: -0.1700

Data Index: 2154709 (Open Web Text)

Max Activating Token Index: 865

Text #19

Max Range: 6.0313. Min Range: -6.0313

Max Act: 5.7876. Min Act: -0.1700

Data Index: 2893839 (Open Web Text)

Max Activating Token Index: 410