Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2490 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.3241. Min Range: -4.3241

Max Act: 4.3241. Min Act: -0.1700

Data Index: 8471779 (Open Web Text)

Max Activating Token Index: 977

Text #1

Max Range: 4.3241. Min Range: -4.3241

Max Act: 4.2330. Min Act: -0.1700

Data Index: 321060 (Open Web Text)

Max Activating Token Index: 618

Text #2

Max Range: 4.3241. Min Range: -4.3241

Max Act: 4.1751. Min Act: -0.1700

Data Index: 5106250 (Open Web Text)

Max Activating Token Index: 218

Text #3

Max Range: 4.3241. Min Range: -4.3241

Max Act: 4.1604. Min Act: -0.1700

Data Index: 5477042 (Open Web Text)

Max Activating Token Index: 540

Text #4

Max Range: 4.3241. Min Range: -4.3241

Max Act: 4.1365. Min Act: -0.1700

Data Index: 6903197 (Open Web Text)

Max Activating Token Index: 1017

Text #5

Max Range: 4.3241. Min Range: -4.3241

Max Act: 4.1202. Min Act: -0.1700

Data Index: 2068646 (Open Web Text)

Max Activating Token Index: 458

Text #6

Max Range: 4.3241. Min Range: -4.3241

Max Act: 4.0598. Min Act: -0.1700

Data Index: 8312377 (Open Web Text)

Max Activating Token Index: 415

Text #7

Max Range: 4.3241. Min Range: -4.3241

Max Act: 4.1055. Min Act: -0.1700

Data Index: 1393224 (Open Web Text)

Max Activating Token Index: 643

Text #8

Max Range: 4.3241. Min Range: -4.3241

Max Act: 4.0956. Min Act: -0.1700

Data Index: 7366546 (Open Web Text)

Max Activating Token Index: 691

Text #9

Max Range: 4.3241. Min Range: -4.3241

Max Act: 4.1100. Min Act: -0.1700

Data Index: 760047 (Open Web Text)

Max Activating Token Index: 158

Text #10

Max Range: 4.3241. Min Range: -4.3241

Max Act: 4.0967. Min Act: -0.1700

Data Index: 2716930 (Open Web Text)

Max Activating Token Index: 756

Text #11

Max Range: 4.3241. Min Range: -4.3241

Max Act: 4.0512. Min Act: -0.1700

Data Index: 4604382 (Open Web Text)

Max Activating Token Index: 659

Text #12

Max Range: 4.3241. Min Range: -4.3241

Max Act: 4.0646. Min Act: -0.1700

Data Index: 3069876 (Open Web Text)

Max Activating Token Index: 87

Text #13

Max Range: 4.3241. Min Range: -4.3241

Max Act: 4.0654. Min Act: -0.1700

Data Index: 7997531 (Open Web Text)

Max Activating Token Index: 760

Text #14

Max Range: 4.3241. Min Range: -4.3241

Max Act: 4.0988. Min Act: -0.1700

Data Index: 304128 (Open Web Text)

Max Activating Token Index: 910

Text #15

Max Range: 4.3241. Min Range: -4.3241

Max Act: 4.1006. Min Act: -0.1700

Data Index: 3420736 (Open Web Text)

Max Activating Token Index: 362

Text #16

Max Range: 4.3241. Min Range: -4.3241

Max Act: 4.0508. Min Act: -0.1700

Data Index: 4741297 (Open Web Text)

Max Activating Token Index: 105

Text #17

Max Range: 4.3241. Min Range: -4.3241

Max Act: 4.0357. Min Act: -0.1700

Data Index: 1288416 (Open Web Text)

Max Activating Token Index: 435

Text #18

Max Range: 4.3241. Min Range: -4.3241

Max Act: 4.0326. Min Act: -0.1700

Data Index: 1947135 (Open Web Text)

Max Activating Token Index: 369

Text #19

Max Range: 4.3241. Min Range: -4.3241

Max Act: 4.0422. Min Act: -0.1700

Data Index: 5768964 (Open Web Text)

Max Activating Token Index: 385