Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1526 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 6.6266. Min Range: -6.6266

Max Act: 6.6266. Min Act: -0.1700

Data Index: 5767625 (Open Web Text)

Max Activating Token Index: 509

Text #1

Max Range: 6.6266. Min Range: -6.6266

Max Act: 6.4773. Min Act: -0.1700

Data Index: 2342134 (Open Web Text)

Max Activating Token Index: 718

Text #2

Max Range: 6.6266. Min Range: -6.6266

Max Act: 6.3671. Min Act: -0.1700

Data Index: 4461298 (Open Web Text)

Max Activating Token Index: 544

Text #3

Max Range: 6.6266. Min Range: -6.6266

Max Act: 6.3592. Min Act: -0.1700

Data Index: 7318805 (Open Web Text)

Max Activating Token Index: 989

Text #4

Max Range: 6.6266. Min Range: -6.6266

Max Act: 6.2591. Min Act: -0.1700

Data Index: 2414031 (Open Web Text)

Max Activating Token Index: 266

Text #5

Max Range: 6.6266. Min Range: -6.6266

Max Act: 6.2882. Min Act: -0.1700

Data Index: 1688333 (Open Web Text)

Max Activating Token Index: 497

Text #6

Max Range: 6.6266. Min Range: -6.6266

Max Act: 6.2282. Min Act: -0.1700

Data Index: 80268 (Open Web Text)

Max Activating Token Index: 548

Text #7

Max Range: 6.6266. Min Range: -6.6266

Max Act: 6.2066. Min Act: -0.1700

Data Index: 4484705 (Open Web Text)

Max Activating Token Index: 496

Text #8

Max Range: 6.6266. Min Range: -6.6266

Max Act: 6.2223. Min Act: -0.1700

Data Index: 3178329 (Open Web Text)

Max Activating Token Index: 1021

Text #9

Max Range: 6.6266. Min Range: -6.6266

Max Act: 6.2055. Min Act: -0.1700

Data Index: 1412536 (Open Web Text)

Max Activating Token Index: 553

Text #10

Max Range: 6.6266. Min Range: -6.6266

Max Act: 6.1799. Min Act: -0.1700

Data Index: 5075275 (Open Web Text)

Max Activating Token Index: 847

Text #11

Max Range: 6.6266. Min Range: -6.6266

Max Act: 6.1908. Min Act: -0.1700

Data Index: 3966244 (Open Web Text)

Max Activating Token Index: 899

Text #12

Max Range: 6.6266. Min Range: -6.6266

Max Act: 6.1990. Min Act: -0.1700

Data Index: 6878468 (Open Web Text)

Max Activating Token Index: 638

Text #13

Max Range: 6.6266. Min Range: -6.6266

Max Act: 6.1569. Min Act: -0.1700

Data Index: 6791080 (Open Web Text)

Max Activating Token Index: 442

Text #14

Max Range: 6.6266. Min Range: -6.6266

Max Act: 6.1266. Min Act: -0.1700

Data Index: 4651830 (Open Web Text)

Max Activating Token Index: 440

Text #15

Max Range: 6.6266. Min Range: -6.6266

Max Act: 6.1464. Min Act: -0.1700

Data Index: 1870287 (Open Web Text)

Max Activating Token Index: 906

Text #16

Max Range: 6.6266. Min Range: -6.6266

Max Act: 6.0944. Min Act: -0.1700

Data Index: 1366423 (Open Web Text)

Max Activating Token Index: 883

Text #17

Max Range: 6.6266. Min Range: -6.6266

Max Act: 6.0822. Min Act: -0.1700

Data Index: 3326575 (Open Web Text)

Max Activating Token Index: 605

Text #18

Max Range: 6.6266. Min Range: -6.6266

Max Act: 6.0679. Min Act: -0.1700

Data Index: 3379466 (Open Web Text)

Max Activating Token Index: 342

Text #19

Max Range: 6.6266. Min Range: -6.6266

Max Act: 6.0723. Min Act: -0.1700

Data Index: 4260777 (Open Web Text)

Max Activating Token Index: 545