Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1366 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.8139. Min Range: -4.8139

Max Act: 4.8139. Min Act: -0.1700

Data Index: 4371244 (Open Web Text)

Max Activating Token Index: 990

Text #1

Max Range: 4.8139. Min Range: -4.8139

Max Act: 4.7912. Min Act: -0.1700

Data Index: 7208698 (Open Web Text)

Max Activating Token Index: 638

Text #2

Max Range: 4.8139. Min Range: -4.8139

Max Act: 4.7475. Min Act: -0.1700

Data Index: 7281457 (Open Web Text)

Max Activating Token Index: 463

Text #3

Max Range: 4.8139. Min Range: -4.8139

Max Act: 4.6510. Min Act: -0.1700

Data Index: 287497 (Open Web Text)

Max Activating Token Index: 34

Text #4

Max Range: 4.8139. Min Range: -4.8139

Max Act: 4.6618. Min Act: -0.1700

Data Index: 1471452 (Open Web Text)

Max Activating Token Index: 901

Text #5

Max Range: 4.8139. Min Range: -4.8139

Max Act: 4.6439. Min Act: -0.1700

Data Index: 6894038 (Open Web Text)

Max Activating Token Index: 372

Text #6

Max Range: 4.8139. Min Range: -4.8139

Max Act: 4.6469. Min Act: -0.1700

Data Index: 6462700 (Open Web Text)

Max Activating Token Index: 844

Text #7

Max Range: 4.8139. Min Range: -4.8139

Max Act: 4.6378. Min Act: -0.1700

Data Index: 4246187 (Open Web Text)

Max Activating Token Index: 747

Text #8

Max Range: 4.8139. Min Range: -4.8139

Max Act: 4.6224. Min Act: -0.1700

Data Index: 15921 (Open Web Text)

Max Activating Token Index: 1005

Text #9

Max Range: 4.8139. Min Range: -4.8139

Max Act: 4.6454. Min Act: -0.1700

Data Index: 1065435 (Open Web Text)

Max Activating Token Index: 359

Text #10

Max Range: 4.8139. Min Range: -4.8139

Max Act: 4.6148. Min Act: -0.1700

Data Index: 1530475 (Open Web Text)

Max Activating Token Index: 164

Text #11

Max Range: 4.8139. Min Range: -4.8139

Max Act: 4.6406. Min Act: -0.1700

Data Index: 3290785 (Open Web Text)

Max Activating Token Index: 932

Text #12

Max Range: 4.8139. Min Range: -4.8139

Max Act: 4.5999. Min Act: -0.1700

Data Index: 807431 (Open Web Text)

Max Activating Token Index: 355

Text #13

Max Range: 4.8139. Min Range: -4.8139

Max Act: 4.6187. Min Act: -0.1700

Data Index: 2516148 (Open Web Text)

Max Activating Token Index: 307

Text #14

Max Range: 4.8139. Min Range: -4.8139

Max Act: 4.5948. Min Act: -0.1700

Data Index: 8682058 (Open Web Text)

Max Activating Token Index: 984

Text #15

Max Range: 4.8139. Min Range: -4.8139

Max Act: 4.5923. Min Act: -0.1700

Data Index: 5045194 (Open Web Text)

Max Activating Token Index: 978

Text #16

Max Range: 4.8139. Min Range: -4.8139

Max Act: 4.6192. Min Act: -0.1700

Data Index: 4985938 (Open Web Text)

Max Activating Token Index: 806

Text #17

Max Range: 4.8139. Min Range: -4.8139

Max Act: 4.5956. Min Act: -0.1700

Data Index: 1611315 (Open Web Text)

Max Activating Token Index: 696

Text #18

Max Range: 4.8139. Min Range: -4.8139

Max Act: 4.5862. Min Act: -0.1700

Data Index: 4379621 (Open Web Text)

Max Activating Token Index: 800

Text #19

Max Range: 4.8139. Min Range: -4.8139

Max Act: 4.5968. Min Act: -0.1700

Data Index: 7714287 (Open Web Text)

Max Activating Token Index: 11