Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1533 in Layer 4

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.1951. Min Range: -3.1951

Max Act: 3.1951. Min Act: -0.1700

Data Index: 700474 (Open Web Text)

Max Activating Token Index: 84

Text #1

Max Range: 3.1951. Min Range: -3.1951

Max Act: 3.1873. Min Act: -0.1700

Data Index: 6218019 (Open Web Text)

Max Activating Token Index: 747

Text #2

Max Range: 3.1951. Min Range: -3.1951

Max Act: 3.1316. Min Act: -0.1700

Data Index: 5725481 (Open Web Text)

Max Activating Token Index: 88

Text #3

Max Range: 3.1951. Min Range: -3.1951

Max Act: 3.1219. Min Act: -0.1700

Data Index: 5775535 (Open Web Text)

Max Activating Token Index: 19

Text #4

Max Range: 3.1951. Min Range: -3.1951

Max Act: 3.0808. Min Act: -0.1700

Data Index: 1897801 (Open Web Text)

Max Activating Token Index: 289

Text #5

Max Range: 3.1951. Min Range: -3.1951

Max Act: 3.0442. Min Act: -0.1700

Data Index: 1041934 (Open Web Text)

Max Activating Token Index: 742

Text #6

Max Range: 3.1951. Min Range: -3.1951

Max Act: 3.0527. Min Act: -0.1700

Data Index: 4629716 (Open Web Text)

Max Activating Token Index: 553

Text #7

Max Range: 3.1951. Min Range: -3.1951

Max Act: 3.0423. Min Act: -0.1700

Data Index: 7162973 (Open Web Text)

Max Activating Token Index: 266

Text #8

Max Range: 3.1951. Min Range: -3.1951

Max Act: 3.0081. Min Act: -0.1700

Data Index: 450777 (Open Web Text)

Max Activating Token Index: 54

Text #9

Max Range: 3.1951. Min Range: -3.1951

Max Act: 3.0107. Min Act: -0.1700

Data Index: 6222776 (Open Web Text)

Max Activating Token Index: 212

Text #10

Max Range: 3.1951. Min Range: -3.1951

Max Act: 3.0316. Min Act: -0.1700

Data Index: 879529 (Open Web Text)

Max Activating Token Index: 800

Text #11

Max Range: 3.1951. Min Range: -3.1951

Max Act: 3.0022. Min Act: -0.1700

Data Index: 170649 (Open Web Text)

Max Activating Token Index: 60

Text #12

Max Range: 3.1951. Min Range: -3.1951

Max Act: 2.9748. Min Act: -0.1700

Data Index: 8771260 (Open Web Text)

Max Activating Token Index: 423

Text #13

Max Range: 3.1951. Min Range: -3.1951

Max Act: 2.9701. Min Act: -0.1700

Data Index: 7348296 (Open Web Text)

Max Activating Token Index: 216

Text #14

Max Range: 3.1951. Min Range: -3.1951

Max Act: 2.9454. Min Act: -0.1700

Data Index: 6082652 (Open Web Text)

Max Activating Token Index: 731

Text #15

Max Range: 3.1951. Min Range: -3.1951

Max Act: 2.9736. Min Act: -0.1700

Data Index: 5721016 (Open Web Text)

Max Activating Token Index: 54

Text #16

Max Range: 3.1951. Min Range: -3.1951

Max Act: 2.9288. Min Act: -0.1700

Data Index: 2199127 (Open Web Text)

Max Activating Token Index: 49

Text #17

Max Range: 3.1951. Min Range: -3.1951

Max Act: 2.9547. Min Act: -0.1700

Data Index: 6832273 (Open Web Text)

Max Activating Token Index: 379

Text #18

Max Range: 3.1951. Min Range: -3.1951

Max Act: 2.9543. Min Act: -0.1700

Data Index: 7444665 (Open Web Text)

Max Activating Token Index: 234

Text #19

Max Range: 3.1951. Min Range: -3.1951

Max Act: 2.9592. Min Act: -0.1700

Data Index: 52841 (Open Web Text)

Max Activating Token Index: 892