Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1616 in Layer 2

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.0230. Min Range: -4.0230

Max Act: 4.0230. Min Act: -0.1700

Data Index: 6507577 (Open Web Text)

Max Activating Token Index: 4

Text #1

Max Range: 4.0230. Min Range: -4.0230

Max Act: 3.9841. Min Act: -0.1700

Data Index: 5349064 (Open Web Text)

Max Activating Token Index: 480

Text #2

Max Range: 4.0230. Min Range: -4.0230

Max Act: 3.9895. Min Act: -0.1700

Data Index: 8222113 (Open Web Text)

Max Activating Token Index: 384

Text #3

Max Range: 4.0230. Min Range: -4.0230

Max Act: 3.9911. Min Act: -0.1700

Data Index: 4684916 (Open Web Text)

Max Activating Token Index: 673

Text #4

Max Range: 4.0230. Min Range: -4.0230

Max Act: 4.0117. Min Act: -0.1700

Data Index: 3001866 (Open Web Text)

Max Activating Token Index: 261

Text #5

Max Range: 4.0230. Min Range: -4.0230

Max Act: 3.9422. Min Act: -0.1700

Data Index: 7575059 (Open Web Text)

Max Activating Token Index: 719

Text #6

Max Range: 4.0230. Min Range: -4.0230

Max Act: 3.9325. Min Act: -0.1700

Data Index: 3200163 (Open Web Text)

Max Activating Token Index: 451

Text #7

Max Range: 4.0230. Min Range: -4.0230

Max Act: 3.9157. Min Act: -0.1700

Data Index: 6734054 (Open Web Text)

Max Activating Token Index: 536

Text #8

Max Range: 4.0230. Min Range: -4.0230

Max Act: 3.9013. Min Act: -0.1700

Data Index: 5446563 (Open Web Text)

Max Activating Token Index: 224

Text #9

Max Range: 4.0230. Min Range: -4.0230

Max Act: 3.8974. Min Act: -0.1700

Data Index: 7171910 (Open Web Text)

Max Activating Token Index: 383

Text #10

Max Range: 4.0230. Min Range: -4.0230

Max Act: 3.9525. Min Act: -0.1700

Data Index: 680031 (Open Web Text)

Max Activating Token Index: 851

Text #11

Max Range: 4.0230. Min Range: -4.0230

Max Act: 3.9486. Min Act: -0.1700

Data Index: 2741614 (Open Web Text)

Max Activating Token Index: 628

Text #12

Max Range: 4.0230. Min Range: -4.0230

Max Act: 3.8912. Min Act: -0.1700

Data Index: 5551780 (Open Web Text)

Max Activating Token Index: 434

Text #13

Max Range: 4.0230. Min Range: -4.0230

Max Act: 3.8907. Min Act: -0.1700

Data Index: 5659886 (Open Web Text)

Max Activating Token Index: 95

Text #14

Max Range: 4.0230. Min Range: -4.0230

Max Act: 3.8202. Min Act: -0.1700

Data Index: 4689218 (Open Web Text)

Max Activating Token Index: 673

Text #15

Max Range: 4.0230. Min Range: -4.0230

Max Act: 3.8387. Min Act: -0.1700

Data Index: 6629712 (Open Web Text)

Max Activating Token Index: 234

Text #16

Max Range: 4.0230. Min Range: -4.0230

Max Act: 3.8845. Min Act: -0.1700

Data Index: 7089074 (Open Web Text)

Max Activating Token Index: 835

Text #17

Max Range: 4.0230. Min Range: -4.0230

Max Act: 3.8549. Min Act: -0.1700

Data Index: 3522222 (Open Web Text)

Max Activating Token Index: 301

Text #18

Max Range: 4.0230. Min Range: -4.0230

Max Act: 3.8501. Min Act: -0.1700

Data Index: 105453 (Open Web Text)

Max Activating Token Index: 4

Text #19

Max Range: 4.0230. Min Range: -4.0230

Max Act: 3.8548. Min Act: -0.1700

Data Index: 3388937 (Open Web Text)

Max Activating Token Index: 711