Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2361 in Layer 2

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 2.2744. Min Range: -2.2744

Max Act: 2.2727. Min Act: -0.1700

Data Index: 1547154 (Open Web Text)

Max Activating Token Index: 323

Text #1

Max Range: 2.2744. Min Range: -2.2744

Max Act: 2.2744. Min Act: -0.1700

Data Index: 8794602 (Open Web Text)

Max Activating Token Index: 429

Text #2

Max Range: 2.2744. Min Range: -2.2744

Max Act: 2.2397. Min Act: -0.1700

Data Index: 8821602 (Open Web Text)

Max Activating Token Index: 599

Text #3

Max Range: 2.2744. Min Range: -2.2744

Max Act: 2.2398. Min Act: -0.1700

Data Index: 185410 (Open Web Text)

Max Activating Token Index: 604

Text #4

Max Range: 2.2744. Min Range: -2.2744

Max Act: 2.2342. Min Act: -0.1700

Data Index: 345147 (Open Web Text)

Max Activating Token Index: 732

Text #5

Max Range: 2.2744. Min Range: -2.2744

Max Act: 2.1996. Min Act: -0.1700

Data Index: 8794624 (Open Web Text)

Max Activating Token Index: 270

Text #6

Max Range: 2.2744. Min Range: -2.2744

Max Act: 2.1978. Min Act: -0.1700

Data Index: 7097450 (Open Web Text)

Max Activating Token Index: 536

Text #7

Max Range: 2.2744. Min Range: -2.2744

Max Act: 2.2188. Min Act: -0.1700

Data Index: 8069108 (Open Web Text)

Max Activating Token Index: 520

Text #8

Max Range: 2.2744. Min Range: -2.2744

Max Act: 2.2070. Min Act: -0.1700

Data Index: 8177864 (Open Web Text)

Max Activating Token Index: 628

Text #9

Max Range: 2.2744. Min Range: -2.2744

Max Act: 2.1926. Min Act: -0.1700

Data Index: 4909334 (Open Web Text)

Max Activating Token Index: 544

Text #10

Max Range: 2.2744. Min Range: -2.2744

Max Act: 2.2168. Min Act: -0.1700

Data Index: 2164818 (Open Web Text)

Max Activating Token Index: 574

Text #11

Max Range: 2.2744. Min Range: -2.2744

Max Act: 2.1915. Min Act: -0.1700

Data Index: 3074218 (Open Web Text)

Max Activating Token Index: 658

Text #12

Max Range: 2.2744. Min Range: -2.2744

Max Act: 2.1783. Min Act: -0.1700

Data Index: 4796518 (Open Web Text)

Max Activating Token Index: 613

Text #13

Max Range: 2.2744. Min Range: -2.2744

Max Act: 2.1872. Min Act: -0.1700

Data Index: 6131506 (Open Web Text)

Max Activating Token Index: 697

Text #14

Max Range: 2.2744. Min Range: -2.2744

Max Act: 2.1759. Min Act: -0.1700

Data Index: 2551069 (Open Web Text)

Max Activating Token Index: 348

Text #15

Max Range: 2.2744. Min Range: -2.2744

Max Act: 2.1766. Min Act: -0.1700

Data Index: 2551070 (Open Web Text)

Max Activating Token Index: 550

Text #16

Max Range: 2.2744. Min Range: -2.2744

Max Act: 2.1616. Min Act: -0.1700

Data Index: 185411 (Open Web Text)

Max Activating Token Index: 683

Text #17

Max Range: 2.2744. Min Range: -2.2744

Max Act: 2.1487. Min Act: -0.1700

Data Index: 2116786 (Open Web Text)

Max Activating Token Index: 585

Text #18

Max Range: 2.2744. Min Range: -2.2744

Max Act: 2.1409. Min Act: -0.1700

Data Index: 5897915 (Open Web Text)

Max Activating Token Index: 499

Text #19

Max Range: 2.2744. Min Range: -2.2744

Max Act: 2.1582. Min Act: -0.1700

Data Index: 4909301 (Open Web Text)

Max Activating Token Index: 997