Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 928 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 5.3434. Min Range: -5.3434

Max Act: 5.3434. Min Act: -0.1700

Data Index: 5403019 (Open Web Text)

Max Activating Token Index: 31

Text #1

Max Range: 5.3434. Min Range: -5.3434

Max Act: 5.2691. Min Act: -0.1700

Data Index: 3164089 (Open Web Text)

Max Activating Token Index: 198

Text #2

Max Range: 5.3434. Min Range: -5.3434

Max Act: 5.2339. Min Act: -0.1700

Data Index: 8166249 (Open Web Text)

Max Activating Token Index: 337

Text #3

Max Range: 5.3434. Min Range: -5.3434

Max Act: 5.2146. Min Act: -0.1700

Data Index: 8754523 (Open Web Text)

Max Activating Token Index: 453

Text #4

Max Range: 5.3434. Min Range: -5.3434

Max Act: 5.2123. Min Act: -0.1700

Data Index: 2374428 (Open Web Text)

Max Activating Token Index: 989

Text #5

Max Range: 5.3434. Min Range: -5.3434

Max Act: 5.2367. Min Act: -0.1700

Data Index: 5422331 (Open Web Text)

Max Activating Token Index: 216

Text #6

Max Range: 5.3434. Min Range: -5.3434

Max Act: 5.2275. Min Act: -0.1700

Data Index: 5266892 (Open Web Text)

Max Activating Token Index: 646

Text #7

Max Range: 5.3434. Min Range: -5.3434

Max Act: 5.1807. Min Act: -0.1700

Data Index: 3941998 (Open Web Text)

Max Activating Token Index: 987

Text #8

Max Range: 5.3434. Min Range: -5.3434

Max Act: 5.2136. Min Act: -0.1700

Data Index: 4512233 (Open Web Text)

Max Activating Token Index: 1018

Text #9

Max Range: 5.3434. Min Range: -5.3434

Max Act: 5.1817. Min Act: -0.1700

Data Index: 7011605 (Open Web Text)

Max Activating Token Index: 839

Text #10

Max Range: 5.3434. Min Range: -5.3434

Max Act: 5.2009. Min Act: -0.1700

Data Index: 6818634 (Open Web Text)

Max Activating Token Index: 671

Text #11

Max Range: 5.3434. Min Range: -5.3434

Max Act: 5.2114. Min Act: -0.1700

Data Index: 1405871 (Open Web Text)

Max Activating Token Index: 997

Text #12

Max Range: 5.3434. Min Range: -5.3434

Max Act: 5.2025. Min Act: -0.1700

Data Index: 1640383 (Open Web Text)

Max Activating Token Index: 476

Text #13

Max Range: 5.3434. Min Range: -5.3434

Max Act: 5.2008. Min Act: -0.1700

Data Index: 6489557 (Open Web Text)

Max Activating Token Index: 183

Text #14

Max Range: 5.3434. Min Range: -5.3434

Max Act: 5.1587. Min Act: -0.1700

Data Index: 5934831 (Open Web Text)

Max Activating Token Index: 950

Text #15

Max Range: 5.3434. Min Range: -5.3434

Max Act: 5.2024. Min Act: -0.1700

Data Index: 8457722 (Open Web Text)

Max Activating Token Index: 616

Text #16

Max Range: 5.3434. Min Range: -5.3434

Max Act: 5.1710. Min Act: -0.1700

Data Index: 8104505 (Open Web Text)

Max Activating Token Index: 280

Text #17

Max Range: 5.3434. Min Range: -5.3434

Max Act: 5.1579. Min Act: -0.1700

Data Index: 3407121 (Open Web Text)

Max Activating Token Index: 622

Text #18

Max Range: 5.3434. Min Range: -5.3434

Max Act: 5.1458. Min Act: -0.1700

Data Index: 2613021 (Open Web Text)

Max Activating Token Index: 994

Text #19

Max Range: 5.3434. Min Range: -5.3434

Max Act: 5.1688. Min Act: -0.1700

Data Index: 8281538 (Open Web Text)

Max Activating Token Index: 884