Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1960 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.4880. Min Range: -3.4880

Max Act: 3.4880. Min Act: -0.1700

Data Index: 1543734 (Open Web Text)

Max Activating Token Index: 16

Text #1

Max Range: 3.4880. Min Range: -3.4880

Max Act: 3.4870. Min Act: -0.1700

Data Index: 2759485 (Open Web Text)

Max Activating Token Index: 897

Text #2

Max Range: 3.4880. Min Range: -3.4880

Max Act: 3.4707. Min Act: -0.1700

Data Index: 4085920 (Open Web Text)

Max Activating Token Index: 737

Text #3

Max Range: 3.4880. Min Range: -3.4880

Max Act: 3.4176. Min Act: -0.1700

Data Index: 2729876 (Open Web Text)

Max Activating Token Index: 378

Text #4

Max Range: 3.4880. Min Range: -3.4880

Max Act: 3.4040. Min Act: -0.1700

Data Index: 3725038 (Open Web Text)

Max Activating Token Index: 353

Text #5

Max Range: 3.4880. Min Range: -3.4880

Max Act: 3.4430. Min Act: -0.1700

Data Index: 7413150 (Open Web Text)

Max Activating Token Index: 1017

Text #6

Max Range: 3.4880. Min Range: -3.4880

Max Act: 3.4240. Min Act: -0.1700

Data Index: 4794996 (Open Web Text)

Max Activating Token Index: 622

Text #7

Max Range: 3.4880. Min Range: -3.4880

Max Act: 3.3979. Min Act: -0.1700

Data Index: 34913 (Open Web Text)

Max Activating Token Index: 652

Text #8

Max Range: 3.4880. Min Range: -3.4880

Max Act: 3.3571. Min Act: -0.1700

Data Index: 1466205 (Open Web Text)

Max Activating Token Index: 10

Text #9

Max Range: 3.4880. Min Range: -3.4880

Max Act: 3.3747. Min Act: -0.1700

Data Index: 6323935 (Open Web Text)

Max Activating Token Index: 59

Text #10

Max Range: 3.4880. Min Range: -3.4880

Max Act: 3.3696. Min Act: -0.1700

Data Index: 1001108 (Open Web Text)

Max Activating Token Index: 184

Text #11

Max Range: 3.4880. Min Range: -3.4880

Max Act: 3.3375. Min Act: -0.1700

Data Index: 2290908 (Open Web Text)

Max Activating Token Index: 970

Text #12

Max Range: 3.4880. Min Range: -3.4880

Max Act: 3.3568. Min Act: -0.1700

Data Index: 2736137 (Open Web Text)

Max Activating Token Index: 10

Text #13

Max Range: 3.4880. Min Range: -3.4880

Max Act: 3.3558. Min Act: -0.1700

Data Index: 3978203 (Open Web Text)

Max Activating Token Index: 715

Text #14

Max Range: 3.4880. Min Range: -3.4880

Max Act: 3.3439. Min Act: -0.1700

Data Index: 1365084 (Open Web Text)

Max Activating Token Index: 675

Text #15

Max Range: 3.4880. Min Range: -3.4880

Max Act: 3.3448. Min Act: -0.1700

Data Index: 4284335 (Open Web Text)

Max Activating Token Index: 476

Text #16

Max Range: 3.4880. Min Range: -3.4880

Max Act: 3.3467. Min Act: -0.1700

Data Index: 3122584 (Open Web Text)

Max Activating Token Index: 681

Text #17

Max Range: 3.4880. Min Range: -3.4880

Max Act: 3.3598. Min Act: -0.1700

Data Index: 4408178 (Open Web Text)

Max Activating Token Index: 523

Text #18

Max Range: 3.4880. Min Range: -3.4880

Max Act: 3.3401. Min Act: -0.1700

Data Index: 1461262 (Open Web Text)

Max Activating Token Index: 541

Text #19

Max Range: 3.4880. Min Range: -3.4880

Max Act: 3.3493. Min Act: -0.1700

Data Index: 4433904 (Open Web Text)

Max Activating Token Index: 915