Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 155 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.6905. Min Range: -3.6905

Max Act: 3.6905. Min Act: -0.1700

Data Index: 7164651 (Open Web Text)

Max Activating Token Index: 585

Text #1

Max Range: 3.6905. Min Range: -3.6905

Max Act: 3.6314. Min Act: -0.1700

Data Index: 6366282 (Open Web Text)

Max Activating Token Index: 200

Text #2

Max Range: 3.6905. Min Range: -3.6905

Max Act: 3.4698. Min Act: -0.1700

Data Index: 26116 (Open Web Text)

Max Activating Token Index: 227

Text #3

Max Range: 3.6905. Min Range: -3.6905

Max Act: 3.2534. Min Act: -0.1700

Data Index: 8055177 (Open Web Text)

Max Activating Token Index: 435

Text #4

Max Range: 3.6905. Min Range: -3.6905

Max Act: 3.1469. Min Act: -0.1700

Data Index: 6306916 (Open Web Text)

Max Activating Token Index: 740

Text #5

Max Range: 3.6905. Min Range: -3.6905

Max Act: 3.1606. Min Act: -0.1700

Data Index: 8555933 (Open Web Text)

Max Activating Token Index: 874

Text #6

Max Range: 3.6905. Min Range: -3.6905

Max Act: 3.0718. Min Act: -0.1700

Data Index: 6093214 (Open Web Text)

Max Activating Token Index: 430

Text #7

Max Range: 3.6905. Min Range: -3.6905

Max Act: 3.0515. Min Act: -0.1700

Data Index: 7280226 (Open Web Text)

Max Activating Token Index: 147

Text #8

Max Range: 3.6905. Min Range: -3.6905

Max Act: 3.0245. Min Act: -0.1700

Data Index: 6093217 (Open Web Text)

Max Activating Token Index: 124

Text #9

Max Range: 3.6905. Min Range: -3.6905

Max Act: 3.0224. Min Act: -0.1700

Data Index: 6882697 (Open Web Text)

Max Activating Token Index: 341

Text #10

Max Range: 3.6905. Min Range: -3.6905

Max Act: 3.0309. Min Act: -0.1700

Data Index: 4493655 (Open Web Text)

Max Activating Token Index: 472

Text #11

Max Range: 3.6905. Min Range: -3.6905

Max Act: 3.0337. Min Act: -0.1700

Data Index: 594338 (Open Web Text)

Max Activating Token Index: 951

Text #12

Max Range: 3.6905. Min Range: -3.6905

Max Act: 3.0264. Min Act: -0.1700

Data Index: 4708105 (Open Web Text)

Max Activating Token Index: 809

Text #13

Max Range: 3.6905. Min Range: -3.6905

Max Act: 2.9934. Min Act: -0.1700

Data Index: 3935569 (Open Web Text)

Max Activating Token Index: 708

Text #14

Max Range: 3.6905. Min Range: -3.6905

Max Act: 2.9821. Min Act: -0.1700

Data Index: 5747980 (Open Web Text)

Max Activating Token Index: 624

Text #15

Max Range: 3.6905. Min Range: -3.6905

Max Act: 2.9745. Min Act: -0.1700

Data Index: 8264230 (Open Web Text)

Max Activating Token Index: 474

Text #16

Max Range: 3.6905. Min Range: -3.6905

Max Act: 2.9535. Min Act: -0.1700

Data Index: 1512634 (Open Web Text)

Max Activating Token Index: 313

Text #17

Max Range: 3.6905. Min Range: -3.6905

Max Act: 2.9560. Min Act: -0.1700

Data Index: 6124945 (Open Web Text)

Max Activating Token Index: 236

Text #18

Max Range: 3.6905. Min Range: -3.6905

Max Act: 2.9678. Min Act: -0.1700

Data Index: 3399107 (Open Web Text)

Max Activating Token Index: 363

Text #19

Max Range: 3.6905. Min Range: -3.6905

Max Act: 2.9660. Min Act: -0.1700

Data Index: 7129537 (Open Web Text)

Max Activating Token Index: 828