Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 835 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.8046. Min Range: -3.8046

Max Act: 3.7824. Min Act: -0.1700

Data Index: 5545799 (Open Web Text)

Max Activating Token Index: 927

Text #1

Max Range: 3.8046. Min Range: -3.8046

Max Act: 3.8046. Min Act: -0.1700

Data Index: 5028771 (Open Web Text)

Max Activating Token Index: 166

Text #2

Max Range: 3.8046. Min Range: -3.8046

Max Act: 3.7563. Min Act: -0.1700

Data Index: 838939 (Open Web Text)

Max Activating Token Index: 721

Text #3

Max Range: 3.8046. Min Range: -3.8046

Max Act: 3.7443. Min Act: -0.1700

Data Index: 1717560 (Open Web Text)

Max Activating Token Index: 275

Text #4

Max Range: 3.8046. Min Range: -3.8046

Max Act: 3.7095. Min Act: -0.1700

Data Index: 6188899 (Open Web Text)

Max Activating Token Index: 1012

Text #5

Max Range: 3.8046. Min Range: -3.8046

Max Act: 3.6912. Min Act: -0.1700

Data Index: 3675136 (Open Web Text)

Max Activating Token Index: 456

Text #6

Max Range: 3.8046. Min Range: -3.8046

Max Act: 3.6397. Min Act: -0.1700

Data Index: 3249463 (Open Web Text)

Max Activating Token Index: 279

Text #7

Max Range: 3.8046. Min Range: -3.8046

Max Act: 3.6617. Min Act: -0.1700

Data Index: 499296 (Open Web Text)

Max Activating Token Index: 381

Text #8

Max Range: 3.8046. Min Range: -3.8046

Max Act: 3.6320. Min Act: -0.1700

Data Index: 7989539 (Open Web Text)

Max Activating Token Index: 987

Text #9

Max Range: 3.8046. Min Range: -3.8046

Max Act: 3.6255. Min Act: -0.1700

Data Index: 6329764 (Open Web Text)

Max Activating Token Index: 574

Text #10

Max Range: 3.8046. Min Range: -3.8046

Max Act: 3.6011. Min Act: -0.1700

Data Index: 5106464 (Open Web Text)

Max Activating Token Index: 760

Text #11

Max Range: 3.8046. Min Range: -3.8046

Max Act: 3.6103. Min Act: -0.1700

Data Index: 6769434 (Open Web Text)

Max Activating Token Index: 725

Text #12

Max Range: 3.8046. Min Range: -3.8046

Max Act: 3.6286. Min Act: -0.1700

Data Index: 5184376 (Open Web Text)

Max Activating Token Index: 482

Text #13

Max Range: 3.8046. Min Range: -3.8046

Max Act: 3.6233. Min Act: -0.1700

Data Index: 8035312 (Open Web Text)

Max Activating Token Index: 571

Text #14

Max Range: 3.8046. Min Range: -3.8046

Max Act: 3.6246. Min Act: -0.1700

Data Index: 986513 (Open Web Text)

Max Activating Token Index: 703

Text #15

Max Range: 3.8046. Min Range: -3.8046

Max Act: 3.5884. Min Act: -0.1700

Data Index: 2853951 (Open Web Text)

Max Activating Token Index: 708

Text #16

Max Range: 3.8046. Min Range: -3.8046

Max Act: 3.6044. Min Act: -0.1700

Data Index: 7174767 (Open Web Text)

Max Activating Token Index: 449

Text #17

Max Range: 3.8046. Min Range: -3.8046

Max Act: 3.5905. Min Act: -0.1700

Data Index: 2530625 (Open Web Text)

Max Activating Token Index: 666

Text #18

Max Range: 3.8046. Min Range: -3.8046

Max Act: 3.5940. Min Act: -0.1700

Data Index: 7969278 (Open Web Text)

Max Activating Token Index: 578

Text #19

Max Range: 3.8046. Min Range: -3.8046

Max Act: 3.5790. Min Act: -0.1700

Data Index: 154511 (Open Web Text)

Max Activating Token Index: 881