Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1312 in Layer 2

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.7871. Min Range: -3.7871

Max Act: 3.7871. Min Act: -0.1700

Data Index: 1659809 (Open Web Text)

Max Activating Token Index: 815

Text #1

Max Range: 3.7871. Min Range: -3.7871

Max Act: 3.6178. Min Act: -0.1700

Data Index: 3501466 (Open Web Text)

Max Activating Token Index: 625

Text #2

Max Range: 3.7871. Min Range: -3.7871

Max Act: 3.4903. Min Act: -0.1700

Data Index: 7904343 (Open Web Text)

Max Activating Token Index: 752

Text #3

Max Range: 3.7871. Min Range: -3.7871

Max Act: 3.4568. Min Act: -0.1699

Data Index: 7819507 (Open Web Text)

Max Activating Token Index: 161

Text #4

Max Range: 3.7871. Min Range: -3.7871

Max Act: 3.4262. Min Act: -0.1700

Data Index: 1081895 (Open Web Text)

Max Activating Token Index: 336

Text #5

Max Range: 3.7871. Min Range: -3.7871

Max Act: 3.4202. Min Act: -0.1700

Data Index: 3974422 (Open Web Text)

Max Activating Token Index: 322

Text #6

Max Range: 3.7871. Min Range: -3.7871

Max Act: 3.4229. Min Act: -0.1700

Data Index: 2040190 (Open Web Text)

Max Activating Token Index: 613

Text #7

Max Range: 3.7871. Min Range: -3.7871

Max Act: 3.3326. Min Act: -0.1700

Data Index: 7836144 (Open Web Text)

Max Activating Token Index: 833

Text #8

Max Range: 3.7871. Min Range: -3.7871

Max Act: 3.3840. Min Act: -0.1700

Data Index: 5826361 (Open Web Text)

Max Activating Token Index: 485

Text #9

Max Range: 3.7871. Min Range: -3.7871

Max Act: 3.3721. Min Act: -0.1700

Data Index: 3631469 (Open Web Text)

Max Activating Token Index: 375

Text #10

Max Range: 3.7871. Min Range: -3.7871

Max Act: 3.4585. Min Act: -0.1699

Data Index: 4997795 (Open Web Text)

Max Activating Token Index: 533

Text #11

Max Range: 3.7871. Min Range: -3.7871

Max Act: 3.1859. Min Act: -0.1700

Data Index: 5195111 (Open Web Text)

Max Activating Token Index: 606

Text #12

Max Range: 3.7871. Min Range: -3.7871

Max Act: 3.3378. Min Act: -0.1700

Data Index: 1823344 (Open Web Text)

Max Activating Token Index: 780

Text #13

Max Range: 3.7871. Min Range: -3.7871

Max Act: 3.4223. Min Act: -0.1700

Data Index: 7806100 (Open Web Text)

Max Activating Token Index: 259

Text #14

Max Range: 3.7871. Min Range: -3.7871

Max Act: 3.3469. Min Act: -0.1700

Data Index: 5024721 (Open Web Text)

Max Activating Token Index: 205

Text #15

Max Range: 3.7871. Min Range: -3.7871

Max Act: 3.3037. Min Act: -0.1700

Data Index: 1176944 (Open Web Text)

Max Activating Token Index: 507

Text #16

Max Range: 3.7871. Min Range: -3.7871

Max Act: 3.3227. Min Act: -0.1700

Data Index: 1282697 (Open Web Text)

Max Activating Token Index: 833

Text #17

Max Range: 3.7871. Min Range: -3.7871

Max Act: 3.3318. Min Act: -0.1700

Data Index: 3064545 (Open Web Text)

Max Activating Token Index: 271

Text #18

Max Range: 3.7871. Min Range: -3.7871

Max Act: 3.2911. Min Act: -0.1700

Data Index: 1282695 (Open Web Text)

Max Activating Token Index: 188

Text #19

Max Range: 3.7871. Min Range: -3.7871

Max Act: 3.2353. Min Act: -0.1700

Data Index: 2894939 (Open Web Text)

Max Activating Token Index: 305