Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2781 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.2086. Min Range: -3.2086

Max Act: 3.2086. Min Act: -0.1700

Data Index: 8009225 (Open Web Text)

Max Activating Token Index: 913

Text #1

Max Range: 3.2086. Min Range: -3.2086

Max Act: 3.0371. Min Act: -0.1700

Data Index: 8584908 (Open Web Text)

Max Activating Token Index: 604

Text #2

Max Range: 3.2086. Min Range: -3.2086

Max Act: 3.0215. Min Act: -0.1700

Data Index: 7038252 (Open Web Text)

Max Activating Token Index: 81

Text #3

Max Range: 3.2086. Min Range: -3.2086

Max Act: 2.9858. Min Act: -0.1700

Data Index: 1609919 (Open Web Text)

Max Activating Token Index: 174

Text #4

Max Range: 3.2086. Min Range: -3.2086

Max Act: 2.9583. Min Act: -0.1700

Data Index: 1996013 (Open Web Text)

Max Activating Token Index: 249

Text #5

Max Range: 3.2086. Min Range: -3.2086

Max Act: 2.9513. Min Act: -0.1700

Data Index: 565103 (Open Web Text)

Max Activating Token Index: 990

Text #6

Max Range: 3.2086. Min Range: -3.2086

Max Act: 2.9391. Min Act: -0.1700

Data Index: 643552 (Open Web Text)

Max Activating Token Index: 874

Text #7

Max Range: 3.2086. Min Range: -3.2086

Max Act: 2.9232. Min Act: -0.1700

Data Index: 8773213 (Open Web Text)

Max Activating Token Index: 595

Text #8

Max Range: 3.2086. Min Range: -3.2086

Max Act: 2.9059. Min Act: -0.1700

Data Index: 3312089 (Open Web Text)

Max Activating Token Index: 276

Text #9

Max Range: 3.2086. Min Range: -3.2086

Max Act: 2.8943. Min Act: -0.1700

Data Index: 5047197 (Open Web Text)

Max Activating Token Index: 538

Text #10

Max Range: 3.2086. Min Range: -3.2086

Max Act: 2.8948. Min Act: -0.1700

Data Index: 6536843 (Open Web Text)

Max Activating Token Index: 504

Text #11

Max Range: 3.2086. Min Range: -3.2086

Max Act: 2.8741. Min Act: -0.1700

Data Index: 4265837 (Open Web Text)

Max Activating Token Index: 155

Text #12

Max Range: 3.2086. Min Range: -3.2086

Max Act: 2.8541. Min Act: -0.1700

Data Index: 2060974 (Open Web Text)

Max Activating Token Index: 532

Text #13

Max Range: 3.2086. Min Range: -3.2086

Max Act: 2.8550. Min Act: -0.1700

Data Index: 5032669 (Open Web Text)

Max Activating Token Index: 374

Text #14

Max Range: 3.2086. Min Range: -3.2086

Max Act: 2.8626. Min Act: -0.1700

Data Index: 430213 (Open Web Text)

Max Activating Token Index: 501

Text #15

Max Range: 3.2086. Min Range: -3.2086

Max Act: 2.8690. Min Act: -0.1700

Data Index: 1529950 (Open Web Text)

Max Activating Token Index: 132

Text #16

Max Range: 3.2086. Min Range: -3.2086

Max Act: 2.8607. Min Act: -0.1700

Data Index: 645115 (Open Web Text)

Max Activating Token Index: 293

Text #17

Max Range: 3.2086. Min Range: -3.2086

Max Act: 2.8458. Min Act: -0.1700

Data Index: 8235690 (Open Web Text)

Max Activating Token Index: 475

Text #18

Max Range: 3.2086. Min Range: -3.2086

Max Act: 2.8482. Min Act: -0.1700

Data Index: 6634187 (Open Web Text)

Max Activating Token Index: 256

Text #19

Max Range: 3.2086. Min Range: -3.2086

Max Act: 2.8511. Min Act: -0.1700

Data Index: 1113913 (Open Web Text)

Max Activating Token Index: 641