Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2995 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 5.7432. Min Range: -5.7432

Max Act: 5.7058. Min Act: -0.1700

Data Index: 819881 (Open Web Text)

Max Activating Token Index: 1013

Text #1

Max Range: 5.7432. Min Range: -5.7432

Max Act: 5.7399. Min Act: -0.1700

Data Index: 1768637 (Open Web Text)

Max Activating Token Index: 576

Text #2

Max Range: 5.7432. Min Range: -5.7432

Max Act: 5.7208. Min Act: -0.1700

Data Index: 2122792 (Open Web Text)

Max Activating Token Index: 190

Text #3

Max Range: 5.7432. Min Range: -5.7432

Max Act: 5.7228. Min Act: -0.1700

Data Index: 1164927 (Open Web Text)

Max Activating Token Index: 491

Text #4

Max Range: 5.7432. Min Range: -5.7432

Max Act: 5.7432. Min Act: -0.1700

Data Index: 84526 (Open Web Text)

Max Activating Token Index: 75

Text #5

Max Range: 5.7432. Min Range: -5.7432

Max Act: 5.6691. Min Act: -0.1700

Data Index: 6998437 (Open Web Text)

Max Activating Token Index: 807

Text #6

Max Range: 5.7432. Min Range: -5.7432

Max Act: 5.6598. Min Act: -0.1700

Data Index: 6234298 (Open Web Text)

Max Activating Token Index: 877

Text #7

Max Range: 5.7432. Min Range: -5.7432

Max Act: 5.6623. Min Act: -0.1700

Data Index: 3866197 (Open Web Text)

Max Activating Token Index: 759

Text #8

Max Range: 5.7432. Min Range: -5.7432

Max Act: 5.5912. Min Act: -0.1700

Data Index: 7964819 (Open Web Text)

Max Activating Token Index: 537

Text #9

Max Range: 5.7432. Min Range: -5.7432

Max Act: 5.5983. Min Act: -0.1700

Data Index: 3219670 (Open Web Text)

Max Activating Token Index: 666

Text #10

Max Range: 5.7432. Min Range: -5.7432

Max Act: 5.6095. Min Act: -0.1700

Data Index: 5625653 (Open Web Text)

Max Activating Token Index: 744

Text #11

Max Range: 5.7432. Min Range: -5.7432

Max Act: 5.5584. Min Act: -0.1700

Data Index: 7902795 (Open Web Text)

Max Activating Token Index: 879

Text #12

Max Range: 5.7432. Min Range: -5.7432

Max Act: 5.5438. Min Act: -0.1700

Data Index: 89013 (Open Web Text)

Max Activating Token Index: 618

Text #13

Max Range: 5.7432. Min Range: -5.7432

Max Act: 5.5411. Min Act: -0.1700

Data Index: 1858199 (Open Web Text)

Max Activating Token Index: 437

Text #14

Max Range: 5.7432. Min Range: -5.7432

Max Act: 5.5386. Min Act: -0.1700

Data Index: 8659013 (Open Web Text)

Max Activating Token Index: 475

Text #15

Max Range: 5.7432. Min Range: -5.7432

Max Act: 5.5520. Min Act: -0.1700

Data Index: 5267858 (Open Web Text)

Max Activating Token Index: 780

Text #16

Max Range: 5.7432. Min Range: -5.7432

Max Act: 5.6070. Min Act: -0.1700

Data Index: 2964546 (Open Web Text)

Max Activating Token Index: 549

Text #17

Max Range: 5.7432. Min Range: -5.7432

Max Act: 5.5452. Min Act: -0.1700

Data Index: 2228739 (Open Web Text)

Max Activating Token Index: 914

Text #18

Max Range: 5.7432. Min Range: -5.7432

Max Act: 5.5715. Min Act: -0.1700

Data Index: 3123493 (Open Web Text)

Max Activating Token Index: 219

Text #19

Max Range: 5.7432. Min Range: -5.7432

Max Act: 5.5521. Min Act: -0.1700

Data Index: 2314737 (Open Web Text)

Max Activating Token Index: 598