Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2409 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.9801. Min Range: -3.9801

Max Act: 3.9801. Min Act: -0.1700

Data Index: 5918241 (Open Web Text)

Max Activating Token Index: 403

Text #1

Max Range: 3.9801. Min Range: -3.9801

Max Act: 3.9381. Min Act: -0.1700

Data Index: 3038749 (Open Web Text)

Max Activating Token Index: 377

Text #2

Max Range: 3.9801. Min Range: -3.9801

Max Act: 3.6707. Min Act: -0.1700

Data Index: 4635877 (Open Web Text)

Max Activating Token Index: 424

Text #3

Max Range: 3.9801. Min Range: -3.9801

Max Act: 3.6660. Min Act: -0.1700

Data Index: 904083 (Open Web Text)

Max Activating Token Index: 759

Text #4

Max Range: 3.9801. Min Range: -3.9801

Max Act: 3.6430. Min Act: -0.1700

Data Index: 4237046 (Open Web Text)

Max Activating Token Index: 915

Text #5

Max Range: 3.9801. Min Range: -3.9801

Max Act: 3.5933. Min Act: -0.1700

Data Index: 4975363 (Open Web Text)

Max Activating Token Index: 992

Text #6

Max Range: 3.9801. Min Range: -3.9801

Max Act: 3.6141. Min Act: -0.1700

Data Index: 4124814 (Open Web Text)

Max Activating Token Index: 984

Text #7

Max Range: 3.9801. Min Range: -3.9801

Max Act: 3.5895. Min Act: -0.1700

Data Index: 8415665 (Open Web Text)

Max Activating Token Index: 962

Text #8

Max Range: 3.9801. Min Range: -3.9801

Max Act: 3.6035. Min Act: -0.1700

Data Index: 1327008 (Open Web Text)

Max Activating Token Index: 420

Text #9

Max Range: 3.9801. Min Range: -3.9801

Max Act: 3.5939. Min Act: -0.1700

Data Index: 4761603 (Open Web Text)

Max Activating Token Index: 540

Text #10

Max Range: 3.9801. Min Range: -3.9801

Max Act: 3.5643. Min Act: -0.1700

Data Index: 2329263 (Open Web Text)

Max Activating Token Index: 327

Text #11

Max Range: 3.9801. Min Range: -3.9801

Max Act: 3.6165. Min Act: -0.1700

Data Index: 5485603 (Open Web Text)

Max Activating Token Index: 241

Text #12

Max Range: 3.9801. Min Range: -3.9801

Max Act: 3.5342. Min Act: -0.1700

Data Index: 485428 (Open Web Text)

Max Activating Token Index: 339

Text #13

Max Range: 3.9801. Min Range: -3.9801

Max Act: 3.4905. Min Act: -0.1700

Data Index: 7627589 (Open Web Text)

Max Activating Token Index: 1000

Text #14

Max Range: 3.9801. Min Range: -3.9801

Max Act: 3.5009. Min Act: -0.1700

Data Index: 6937314 (Open Web Text)

Max Activating Token Index: 915

Text #15

Max Range: 3.9801. Min Range: -3.9801

Max Act: 3.4814. Min Act: -0.1700

Data Index: 7709585 (Open Web Text)

Max Activating Token Index: 523

Text #16

Max Range: 3.9801. Min Range: -3.9801

Max Act: 3.4412. Min Act: -0.1700

Data Index: 589404 (Open Web Text)

Max Activating Token Index: 581

Text #17

Max Range: 3.9801. Min Range: -3.9801

Max Act: 3.4789. Min Act: -0.1700

Data Index: 6125699 (Open Web Text)

Max Activating Token Index: 875

Text #18

Max Range: 3.9801. Min Range: -3.9801

Max Act: 3.4260. Min Act: -0.1700

Data Index: 7453713 (Open Web Text)

Max Activating Token Index: 188

Text #19

Max Range: 3.9801. Min Range: -3.9801

Max Act: 3.4365. Min Act: -0.1700

Data Index: 2409300 (Open Web Text)

Max Activating Token Index: 1018