Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2633 in Layer 4

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.2461. Min Range: -3.2461

Max Act: 3.2461. Min Act: -0.1700

Data Index: 7142058 (Open Web Text)

Max Activating Token Index: 292

Text #1

Max Range: 3.2461. Min Range: -3.2461

Max Act: 3.1107. Min Act: -0.1700

Data Index: 7142056 (Open Web Text)

Max Activating Token Index: 566

Text #2

Max Range: 3.2461. Min Range: -3.2461

Max Act: 3.0804. Min Act: -0.1700

Data Index: 7142059 (Open Web Text)

Max Activating Token Index: 390

Text #3

Max Range: 3.2461. Min Range: -3.2461

Max Act: 3.0402. Min Act: -0.1700

Data Index: 7661207 (Open Web Text)

Max Activating Token Index: 844

Text #4

Max Range: 3.2461. Min Range: -3.2461

Max Act: 3.0573. Min Act: -0.1700

Data Index: 7142055 (Open Web Text)

Max Activating Token Index: 942

Text #5

Max Range: 3.2461. Min Range: -3.2461

Max Act: 2.9919. Min Act: -0.1700

Data Index: 7142057 (Open Web Text)

Max Activating Token Index: 979

Text #6

Max Range: 3.2461. Min Range: -3.2461

Max Act: 2.9394. Min Act: -0.1700

Data Index: 7661206 (Open Web Text)

Max Activating Token Index: 577

Text #7

Max Range: 3.2461. Min Range: -3.2461

Max Act: 2.8059. Min Act: -0.1700

Data Index: 7056322 (Open Web Text)

Max Activating Token Index: 516

Text #8

Max Range: 3.2461. Min Range: -3.2461

Max Act: 2.7633. Min Act: -0.1700

Data Index: 8068235 (Open Web Text)

Max Activating Token Index: 826

Text #9

Max Range: 3.2461. Min Range: -3.2461

Max Act: 2.7587. Min Act: -0.1700

Data Index: 1205170 (Open Web Text)

Max Activating Token Index: 602

Text #10

Max Range: 3.2461. Min Range: -3.2461

Max Act: 2.7693. Min Act: -0.1700

Data Index: 2797694 (Open Web Text)

Max Activating Token Index: 798

Text #11

Max Range: 3.2461. Min Range: -3.2461

Max Act: 2.7438. Min Act: -0.1700

Data Index: 1199329 (Open Web Text)

Max Activating Token Index: 964

Text #12

Max Range: 3.2461. Min Range: -3.2461

Max Act: 2.7276. Min Act: -0.1700

Data Index: 8268462 (Open Web Text)

Max Activating Token Index: 912

Text #13

Max Range: 3.2461. Min Range: -3.2461

Max Act: 2.6984. Min Act: -0.1700

Data Index: 6672268 (Open Web Text)

Max Activating Token Index: 729

Text #14

Max Range: 3.2461. Min Range: -3.2461

Max Act: 2.7252. Min Act: -0.1700

Data Index: 5265129 (Open Web Text)

Max Activating Token Index: 975

Text #15

Max Range: 3.2461. Min Range: -3.2461

Max Act: 2.7550. Min Act: -0.1700

Data Index: 1306814 (Open Web Text)

Max Activating Token Index: 996

Text #16

Max Range: 3.2461. Min Range: -3.2461

Max Act: 2.7537. Min Act: -0.1700

Data Index: 1520092 (Open Web Text)

Max Activating Token Index: 407

Text #17

Max Range: 3.2461. Min Range: -3.2461

Max Act: 2.7306. Min Act: -0.1700

Data Index: 7283437 (Open Web Text)

Max Activating Token Index: 207

Text #18

Max Range: 3.2461. Min Range: -3.2461

Max Act: 2.7382. Min Act: -0.1700

Data Index: 7283439 (Open Web Text)

Max Activating Token Index: 1004

Text #19

Max Range: 3.2461. Min Range: -3.2461

Max Act: 2.6894. Min Act: -0.1700

Data Index: 2886138 (Open Web Text)

Max Activating Token Index: 283