Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 94 in Layer 3

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.0602. Min Range: -3.0602

Max Act: 3.0602. Min Act: -0.1700

Data Index: 5578481 (Open Web Text)

Max Activating Token Index: 436

Text #1

Max Range: 3.0602. Min Range: -3.0602

Max Act: 2.9863. Min Act: -0.1700

Data Index: 7118639 (Open Web Text)

Max Activating Token Index: 889

Text #2

Max Range: 3.0602. Min Range: -3.0602

Max Act: 2.9679. Min Act: -0.1700

Data Index: 2373686 (Open Web Text)

Max Activating Token Index: 116

Text #3

Max Range: 3.0602. Min Range: -3.0602

Max Act: 2.9696. Min Act: -0.1700

Data Index: 6921328 (Open Web Text)

Max Activating Token Index: 694

Text #4

Max Range: 3.0602. Min Range: -3.0602

Max Act: 2.9411. Min Act: -0.1700

Data Index: 7866593 (Open Web Text)

Max Activating Token Index: 995

Text #5

Max Range: 3.0602. Min Range: -3.0602

Max Act: 2.9519. Min Act: -0.1700

Data Index: 3964541 (Open Web Text)

Max Activating Token Index: 827

Text #6

Max Range: 3.0602. Min Range: -3.0602

Max Act: 2.9405. Min Act: -0.1700

Data Index: 3741646 (Open Web Text)

Max Activating Token Index: 237

Text #7

Max Range: 3.0602. Min Range: -3.0602

Max Act: 2.9183. Min Act: -0.1700

Data Index: 2110922 (Open Web Text)

Max Activating Token Index: 825

Text #8

Max Range: 3.0602. Min Range: -3.0602

Max Act: 2.9372. Min Act: -0.1700

Data Index: 6466001 (Open Web Text)

Max Activating Token Index: 540

Text #9

Max Range: 3.0602. Min Range: -3.0602

Max Act: 2.9244. Min Act: -0.1700

Data Index: 515163 (Open Web Text)

Max Activating Token Index: 385

Text #10

Max Range: 3.0602. Min Range: -3.0602

Max Act: 2.9200. Min Act: -0.1700

Data Index: 2114234 (Open Web Text)

Max Activating Token Index: 520

Text #11

Max Range: 3.0602. Min Range: -3.0602

Max Act: 2.9155. Min Act: -0.1700

Data Index: 8330966 (Open Web Text)

Max Activating Token Index: 878

Text #12

Max Range: 3.0602. Min Range: -3.0602

Max Act: 2.9077. Min Act: -0.1700

Data Index: 7908989 (Open Web Text)

Max Activating Token Index: 333

Text #13

Max Range: 3.0602. Min Range: -3.0602

Max Act: 2.9266. Min Act: -0.1700

Data Index: 7990542 (Open Web Text)

Max Activating Token Index: 373

Text #14

Max Range: 3.0602. Min Range: -3.0602

Max Act: 2.8840. Min Act: -0.1700

Data Index: 5455833 (Open Web Text)

Max Activating Token Index: 869

Text #15

Max Range: 3.0602. Min Range: -3.0602

Max Act: 2.9050. Min Act: -0.1700

Data Index: 7515178 (Open Web Text)

Max Activating Token Index: 453

Text #16

Max Range: 3.0602. Min Range: -3.0602

Max Act: 2.8720. Min Act: -0.1700

Data Index: 6664705 (Open Web Text)

Max Activating Token Index: 630

Text #17

Max Range: 3.0602. Min Range: -3.0602

Max Act: 2.8803. Min Act: -0.1700

Data Index: 2483108 (Open Web Text)

Max Activating Token Index: 240

Text #18

Max Range: 3.0602. Min Range: -3.0602

Max Act: 2.8788. Min Act: -0.1700

Data Index: 4334458 (Open Web Text)

Max Activating Token Index: 738

Text #19

Max Range: 3.0602. Min Range: -3.0602

Max Act: 2.8895. Min Act: -0.1700

Data Index: 2340111 (Open Web Text)

Max Activating Token Index: 629