Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 627 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.1635. Min Range: -3.1635

Max Act: 3.1635. Min Act: -0.1700

Data Index: 770752 (Open Web Text)

Max Activating Token Index: 196

Text #1

Max Range: 3.1635. Min Range: -3.1635

Max Act: 3.1173. Min Act: -0.1700

Data Index: 986535 (Open Web Text)

Max Activating Token Index: 375

Text #2

Max Range: 3.1635. Min Range: -3.1635

Max Act: 3.1035. Min Act: -0.1700

Data Index: 398352 (Open Web Text)

Max Activating Token Index: 35

Text #3

Max Range: 3.1635. Min Range: -3.1635

Max Act: 3.1157. Min Act: -0.1700

Data Index: 861154 (Open Web Text)

Max Activating Token Index: 891

Text #4

Max Range: 3.1635. Min Range: -3.1635

Max Act: 3.1051. Min Act: -0.1700

Data Index: 1603968 (Open Web Text)

Max Activating Token Index: 563

Text #5

Max Range: 3.1635. Min Range: -3.1635

Max Act: 3.0633. Min Act: -0.1700

Data Index: 5291726 (Open Web Text)

Max Activating Token Index: 329

Text #6

Max Range: 3.1635. Min Range: -3.1635

Max Act: 3.0634. Min Act: -0.1700

Data Index: 1039469 (Open Web Text)

Max Activating Token Index: 773

Text #7

Max Range: 3.1635. Min Range: -3.1635

Max Act: 3.0430. Min Act: -0.1700

Data Index: 4833465 (Open Web Text)

Max Activating Token Index: 84

Text #8

Max Range: 3.1635. Min Range: -3.1635

Max Act: 3.0194. Min Act: -0.1700

Data Index: 124121 (Open Web Text)

Max Activating Token Index: 133

Text #9

Max Range: 3.1635. Min Range: -3.1635

Max Act: 2.9781. Min Act: -0.1700

Data Index: 3326235 (Open Web Text)

Max Activating Token Index: 53

Text #10

Max Range: 3.1635. Min Range: -3.1635

Max Act: 2.9860. Min Act: -0.1700

Data Index: 3587480 (Open Web Text)

Max Activating Token Index: 211

Text #11

Max Range: 3.1635. Min Range: -3.1635

Max Act: 2.9387. Min Act: -0.1700

Data Index: 4222608 (Open Web Text)

Max Activating Token Index: 953

Text #12

Max Range: 3.1635. Min Range: -3.1635

Max Act: 2.9334. Min Act: -0.1700

Data Index: 7761147 (Open Web Text)

Max Activating Token Index: 49

Text #13

Max Range: 3.1635. Min Range: -3.1635

Max Act: 2.9582. Min Act: -0.1700

Data Index: 8363519 (Open Web Text)

Max Activating Token Index: 373

Text #14

Max Range: 3.1635. Min Range: -3.1635

Max Act: 2.9328. Min Act: -0.1700

Data Index: 1373866 (Open Web Text)

Max Activating Token Index: 803

Text #15

Max Range: 3.1635. Min Range: -3.1635

Max Act: 2.9607. Min Act: -0.1700

Data Index: 2580080 (Open Web Text)

Max Activating Token Index: 245

Text #16

Max Range: 3.1635. Min Range: -3.1635

Max Act: 2.9481. Min Act: -0.1700

Data Index: 4410896 (Open Web Text)

Max Activating Token Index: 21

Text #17

Max Range: 3.1635. Min Range: -3.1635

Max Act: 2.9397. Min Act: -0.1700

Data Index: 6826893 (Open Web Text)

Max Activating Token Index: 241

Text #18

Max Range: 3.1635. Min Range: -3.1635

Max Act: 2.9233. Min Act: -0.1700

Data Index: 6809967 (Open Web Text)

Max Activating Token Index: 382

Text #19

Max Range: 3.1635. Min Range: -3.1635

Max Act: 2.9272. Min Act: -0.1700

Data Index: 1239299 (Open Web Text)

Max Activating Token Index: 465