Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2659 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 7.8483. Min Range: -7.8483

Max Act: 7.8483. Min Act: -0.1700

Data Index: 8681690 (Open Web Text)

Max Activating Token Index: 374

Text #1

Max Range: 7.8483. Min Range: -7.8483

Max Act: 7.6712. Min Act: -0.1700

Data Index: 871373 (Open Web Text)

Max Activating Token Index: 939

Text #2

Max Range: 7.8483. Min Range: -7.8483

Max Act: 7.5953. Min Act: -0.1700

Data Index: 4434129 (Open Web Text)

Max Activating Token Index: 64

Text #3

Max Range: 7.8483. Min Range: -7.8483

Max Act: 7.4542. Min Act: -0.1700

Data Index: 5237666 (Open Web Text)

Max Activating Token Index: 767

Text #4

Max Range: 7.8483. Min Range: -7.8483

Max Act: 7.4783. Min Act: -0.1700

Data Index: 7833786 (Open Web Text)

Max Activating Token Index: 912

Text #5

Max Range: 7.8483. Min Range: -7.8483

Max Act: 7.5533. Min Act: -0.1700

Data Index: 1649213 (Open Web Text)

Max Activating Token Index: 451

Text #6

Max Range: 7.8483. Min Range: -7.8483

Max Act: 7.4623. Min Act: -0.1700

Data Index: 7528293 (Open Web Text)

Max Activating Token Index: 181

Text #7

Max Range: 7.8483. Min Range: -7.8483

Max Act: 7.4578. Min Act: -0.1700

Data Index: 1032603 (Open Web Text)

Max Activating Token Index: 524

Text #8

Max Range: 7.8483. Min Range: -7.8483

Max Act: 7.3915. Min Act: -0.1700

Data Index: 1211774 (Open Web Text)

Max Activating Token Index: 669

Text #9

Max Range: 7.8483. Min Range: -7.8483

Max Act: 7.3531. Min Act: -0.1700

Data Index: 6216911 (Open Web Text)

Max Activating Token Index: 426

Text #10

Max Range: 7.8483. Min Range: -7.8483

Max Act: 7.3875. Min Act: -0.1700

Data Index: 1800057 (Open Web Text)

Max Activating Token Index: 555

Text #11

Max Range: 7.8483. Min Range: -7.8483

Max Act: 7.3661. Min Act: -0.1700

Data Index: 5274004 (Open Web Text)

Max Activating Token Index: 395

Text #12

Max Range: 7.8483. Min Range: -7.8483

Max Act: 7.3727. Min Act: -0.1700

Data Index: 8735212 (Open Web Text)

Max Activating Token Index: 814

Text #13

Max Range: 7.8483. Min Range: -7.8483

Max Act: 7.3739. Min Act: -0.1700

Data Index: 3559733 (Open Web Text)

Max Activating Token Index: 741

Text #14

Max Range: 7.8483. Min Range: -7.8483

Max Act: 7.3978. Min Act: -0.1700

Data Index: 8030440 (Open Web Text)

Max Activating Token Index: 211

Text #15

Max Range: 7.8483. Min Range: -7.8483

Max Act: 7.3610. Min Act: -0.1700

Data Index: 3946952 (Open Web Text)

Max Activating Token Index: 759

Text #16

Max Range: 7.8483. Min Range: -7.8483

Max Act: 7.3712. Min Act: -0.1700

Data Index: 6404501 (Open Web Text)

Max Activating Token Index: 797

Text #17

Max Range: 7.8483. Min Range: -7.8483

Max Act: 7.3559. Min Act: -0.1700

Data Index: 2353701 (Open Web Text)

Max Activating Token Index: 351

Text #18

Max Range: 7.8483. Min Range: -7.8483

Max Act: 7.3545. Min Act: -0.1700

Data Index: 6038937 (Open Web Text)

Max Activating Token Index: 358

Text #19

Max Range: 7.8483. Min Range: -7.8483

Max Act: 7.2305. Min Act: -0.1700

Data Index: 358555 (Open Web Text)

Max Activating Token Index: 173