Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1135 in Layer 3

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.2035. Min Range: -4.2035

Max Act: 4.2035. Min Act: -0.1700

Data Index: 5174965 (Open Web Text)

Max Activating Token Index: 103

Text #1

Max Range: 4.2035. Min Range: -4.2035

Max Act: 4.2028. Min Act: -0.1700

Data Index: 8371349 (Open Web Text)

Max Activating Token Index: 1006

Text #2

Max Range: 4.2035. Min Range: -4.2035

Max Act: 4.1778. Min Act: -0.1700

Data Index: 2195252 (Open Web Text)

Max Activating Token Index: 124

Text #3

Max Range: 4.2035. Min Range: -4.2035

Max Act: 4.1253. Min Act: -0.1700

Data Index: 3874068 (Open Web Text)

Max Activating Token Index: 613

Text #4

Max Range: 4.2035. Min Range: -4.2035

Max Act: 4.0899. Min Act: -0.1700

Data Index: 6126730 (Open Web Text)

Max Activating Token Index: 483

Text #5

Max Range: 4.2035. Min Range: -4.2035

Max Act: 4.0603. Min Act: -0.1700

Data Index: 1624356 (Open Web Text)

Max Activating Token Index: 703

Text #6

Max Range: 4.2035. Min Range: -4.2035

Max Act: 4.0348. Min Act: -0.1700

Data Index: 8390816 (Open Web Text)

Max Activating Token Index: 212

Text #7

Max Range: 4.2035. Min Range: -4.2035

Max Act: 4.0711. Min Act: -0.1700

Data Index: 3524136 (Open Web Text)

Max Activating Token Index: 616

Text #8

Max Range: 4.2035. Min Range: -4.2035

Max Act: 4.0381. Min Act: -0.1700

Data Index: 2517165 (Open Web Text)

Max Activating Token Index: 27

Text #9

Max Range: 4.2035. Min Range: -4.2035

Max Act: 3.9919. Min Act: -0.1700

Data Index: 4278565 (Open Web Text)

Max Activating Token Index: 545

Text #10

Max Range: 4.2035. Min Range: -4.2035

Max Act: 4.0146. Min Act: -0.1700

Data Index: 1533755 (Open Web Text)

Max Activating Token Index: 1020

Text #11

Max Range: 4.2035. Min Range: -4.2035

Max Act: 4.0421. Min Act: -0.1700

Data Index: 2611135 (Open Web Text)

Max Activating Token Index: 525

Text #12

Max Range: 4.2035. Min Range: -4.2035

Max Act: 4.0023. Min Act: -0.1700

Data Index: 7556379 (Open Web Text)

Max Activating Token Index: 623

Text #13

Max Range: 4.2035. Min Range: -4.2035

Max Act: 3.9721. Min Act: -0.1700

Data Index: 4358101 (Open Web Text)

Max Activating Token Index: 287

Text #14

Max Range: 4.2035. Min Range: -4.2035

Max Act: 3.9917. Min Act: -0.1700

Data Index: 7233028 (Open Web Text)

Max Activating Token Index: 151

Text #15

Max Range: 4.2035. Min Range: -4.2035

Max Act: 3.9877. Min Act: -0.1700

Data Index: 7431784 (Open Web Text)

Max Activating Token Index: 272

Text #16

Max Range: 4.2035. Min Range: -4.2035

Max Act: 4.0297. Min Act: -0.1700

Data Index: 7975793 (Open Web Text)

Max Activating Token Index: 959

Text #17

Max Range: 4.2035. Min Range: -4.2035

Max Act: 4.0030. Min Act: -0.1700

Data Index: 1417181 (Open Web Text)

Max Activating Token Index: 642

Text #18

Max Range: 4.2035. Min Range: -4.2035

Max Act: 4.0339. Min Act: -0.1700

Data Index: 2951930 (Open Web Text)

Max Activating Token Index: 536

Text #19

Max Range: 4.2035. Min Range: -4.2035

Max Act: 3.9874. Min Act: -0.1700

Data Index: 933344 (Open Web Text)

Max Activating Token Index: 454