Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2275 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.2959. Min Range: -4.2959

Max Act: 4.2959. Min Act: -0.1700

Data Index: 4859129 (Open Web Text)

Max Activating Token Index: 907

Text #1

Max Range: 4.2959. Min Range: -4.2959

Max Act: 3.9470. Min Act: -0.1700

Data Index: 2118892 (Open Web Text)

Max Activating Token Index: 277

Text #2

Max Range: 4.2959. Min Range: -4.2959

Max Act: 3.9075. Min Act: -0.1700

Data Index: 5746732 (Open Web Text)

Max Activating Token Index: 657

Text #3

Max Range: 4.2959. Min Range: -4.2959

Max Act: 3.8500. Min Act: -0.1700

Data Index: 5216943 (Open Web Text)

Max Activating Token Index: 466

Text #4

Max Range: 4.2959. Min Range: -4.2959

Max Act: 3.8052. Min Act: -0.1700

Data Index: 157885 (Open Web Text)

Max Activating Token Index: 1022

Text #5

Max Range: 4.2959. Min Range: -4.2959

Max Act: 3.8116. Min Act: -0.1700

Data Index: 90798 (Open Web Text)

Max Activating Token Index: 24

Text #6

Max Range: 4.2959. Min Range: -4.2959

Max Act: 3.7900. Min Act: -0.1700

Data Index: 4751946 (Open Web Text)

Max Activating Token Index: 53

Text #7

Max Range: 4.2959. Min Range: -4.2959

Max Act: 3.8069. Min Act: -0.1700

Data Index: 3693050 (Open Web Text)

Max Activating Token Index: 313

Text #8

Max Range: 4.2959. Min Range: -4.2959

Max Act: 3.8115. Min Act: -0.1700

Data Index: 4469050 (Open Web Text)

Max Activating Token Index: 215

Text #9

Max Range: 4.2959. Min Range: -4.2959

Max Act: 3.7686. Min Act: -0.1700

Data Index: 4291123 (Open Web Text)

Max Activating Token Index: 879

Text #10

Max Range: 4.2959. Min Range: -4.2959

Max Act: 3.7765. Min Act: -0.1700

Data Index: 3130448 (Open Web Text)

Max Activating Token Index: 668

Text #11

Max Range: 4.2959. Min Range: -4.2959

Max Act: 3.7812. Min Act: -0.1700

Data Index: 4795739 (Open Web Text)

Max Activating Token Index: 714

Text #12

Max Range: 4.2959. Min Range: -4.2959

Max Act: 3.7401. Min Act: -0.1700

Data Index: 6935758 (Open Web Text)

Max Activating Token Index: 753

Text #13

Max Range: 4.2959. Min Range: -4.2959

Max Act: 3.7389. Min Act: -0.1700

Data Index: 8824811 (Open Web Text)

Max Activating Token Index: 896

Text #14

Max Range: 4.2959. Min Range: -4.2959

Max Act: 3.7549. Min Act: -0.1700

Data Index: 8605076 (Open Web Text)

Max Activating Token Index: 978

Text #15

Max Range: 4.2959. Min Range: -4.2959

Max Act: 3.7260. Min Act: -0.1700

Data Index: 7311912 (Open Web Text)

Max Activating Token Index: 901

Text #16

Max Range: 4.2959. Min Range: -4.2959

Max Act: 3.7339. Min Act: -0.1700

Data Index: 3007686 (Open Web Text)

Max Activating Token Index: 304

Text #17

Max Range: 4.2959. Min Range: -4.2959

Max Act: 3.7400. Min Act: -0.1700

Data Index: 4660101 (Open Web Text)

Max Activating Token Index: 218

Text #18

Max Range: 4.2959. Min Range: -4.2959

Max Act: 3.7569. Min Act: -0.1700

Data Index: 2309336 (Open Web Text)

Max Activating Token Index: 376

Text #19

Max Range: 4.2959. Min Range: -4.2959

Max Act: 3.7257. Min Act: -0.1700

Data Index: 5659749 (Open Web Text)

Max Activating Token Index: 250