Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1926 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.3221. Min Range: -3.3221

Max Act: 3.3058. Min Act: -0.1700

Data Index: 3573316 (Open Web Text)

Max Activating Token Index: 984

Text #1

Max Range: 3.3221. Min Range: -3.3221

Max Act: 3.3221. Min Act: -0.1700

Data Index: 5737187 (Open Web Text)

Max Activating Token Index: 958

Text #2

Max Range: 3.3221. Min Range: -3.3221

Max Act: 3.2786. Min Act: -0.1700

Data Index: 5519345 (Open Web Text)

Max Activating Token Index: 906

Text #3

Max Range: 3.3221. Min Range: -3.3221

Max Act: 3.2536. Min Act: -0.1700

Data Index: 6871515 (Open Web Text)

Max Activating Token Index: 922

Text #4

Max Range: 3.3221. Min Range: -3.3221

Max Act: 3.2687. Min Act: -0.1700

Data Index: 8119212 (Open Web Text)

Max Activating Token Index: 519

Text #5

Max Range: 3.3221. Min Range: -3.3221

Max Act: 3.2626. Min Act: -0.1700

Data Index: 2052245 (Open Web Text)

Max Activating Token Index: 244

Text #6

Max Range: 3.3221. Min Range: -3.3221

Max Act: 3.2708. Min Act: -0.1700

Data Index: 2407789 (Open Web Text)

Max Activating Token Index: 1018

Text #7

Max Range: 3.3221. Min Range: -3.3221

Max Act: 3.2592. Min Act: -0.1700

Data Index: 5650500 (Open Web Text)

Max Activating Token Index: 263

Text #8

Max Range: 3.3221. Min Range: -3.3221

Max Act: 3.2447. Min Act: -0.1700

Data Index: 3412248 (Open Web Text)

Max Activating Token Index: 791

Text #9

Max Range: 3.3221. Min Range: -3.3221

Max Act: 3.2663. Min Act: -0.1700

Data Index: 428077 (Open Web Text)

Max Activating Token Index: 861

Text #10

Max Range: 3.3221. Min Range: -3.3221

Max Act: 3.2474. Min Act: -0.1700

Data Index: 6260052 (Open Web Text)

Max Activating Token Index: 263

Text #11

Max Range: 3.3221. Min Range: -3.3221

Max Act: 3.2422. Min Act: -0.1700

Data Index: 6602657 (Open Web Text)

Max Activating Token Index: 607

Text #12

Max Range: 3.3221. Min Range: -3.3221

Max Act: 3.2301. Min Act: -0.1700

Data Index: 1853068 (Open Web Text)

Max Activating Token Index: 805

Text #13

Max Range: 3.3221. Min Range: -3.3221

Max Act: 3.2629. Min Act: -0.1700

Data Index: 6285924 (Open Web Text)

Max Activating Token Index: 764

Text #14

Max Range: 3.3221. Min Range: -3.3221

Max Act: 3.2455. Min Act: -0.1700

Data Index: 2013553 (Open Web Text)

Max Activating Token Index: 712

Text #15

Max Range: 3.3221. Min Range: -3.3221

Max Act: 3.2484. Min Act: -0.1700

Data Index: 3371938 (Open Web Text)

Max Activating Token Index: 694

Text #16

Max Range: 3.3221. Min Range: -3.3221

Max Act: 3.2392. Min Act: -0.1700

Data Index: 3269435 (Open Web Text)

Max Activating Token Index: 994

Text #17

Max Range: 3.3221. Min Range: -3.3221

Max Act: 3.2501. Min Act: -0.1700

Data Index: 3193611 (Open Web Text)

Max Activating Token Index: 675

Text #18

Max Range: 3.3221. Min Range: -3.3221

Max Act: 3.2397. Min Act: -0.1700

Data Index: 5251228 (Open Web Text)

Max Activating Token Index: 625

Text #19

Max Range: 3.3221. Min Range: -3.3221

Max Act: 3.2404. Min Act: -0.1700

Data Index: 2456015 (Open Web Text)

Max Activating Token Index: 789