Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1569 in Layer 4

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.5583. Min Range: -4.5583

Max Act: 4.5583. Min Act: -0.1700

Data Index: 1391621 (Open Web Text)

Max Activating Token Index: 743

Text #1

Max Range: 4.5583. Min Range: -4.5583

Max Act: 4.5364. Min Act: -0.1700

Data Index: 6405931 (Open Web Text)

Max Activating Token Index: 951

Text #2

Max Range: 4.5583. Min Range: -4.5583

Max Act: 4.5424. Min Act: -0.1700

Data Index: 3223353 (Open Web Text)

Max Activating Token Index: 22

Text #3

Max Range: 4.5583. Min Range: -4.5583

Max Act: 4.4773. Min Act: -0.1700

Data Index: 4621034 (Open Web Text)

Max Activating Token Index: 66

Text #4

Max Range: 4.5583. Min Range: -4.5583

Max Act: 4.5272. Min Act: -0.1700

Data Index: 2926136 (Open Web Text)

Max Activating Token Index: 511

Text #5

Max Range: 4.5583. Min Range: -4.5583

Max Act: 4.5075. Min Act: -0.1700

Data Index: 8000763 (Open Web Text)

Max Activating Token Index: 734

Text #6

Max Range: 4.5583. Min Range: -4.5583

Max Act: 4.4728. Min Act: -0.1700

Data Index: 5291803 (Open Web Text)

Max Activating Token Index: 36

Text #7

Max Range: 4.5583. Min Range: -4.5583

Max Act: 4.4765. Min Act: -0.1700

Data Index: 6198212 (Open Web Text)

Max Activating Token Index: 832

Text #8

Max Range: 4.5583. Min Range: -4.5583

Max Act: 4.5080. Min Act: -0.1700

Data Index: 965101 (Open Web Text)

Max Activating Token Index: 494

Text #9

Max Range: 4.5583. Min Range: -4.5583

Max Act: 4.5014. Min Act: -0.1700

Data Index: 2481874 (Open Web Text)

Max Activating Token Index: 16

Text #10

Max Range: 4.5583. Min Range: -4.5583

Max Act: 4.4511. Min Act: -0.1700

Data Index: 2536079 (Open Web Text)

Max Activating Token Index: 770

Text #11

Max Range: 4.5583. Min Range: -4.5583

Max Act: 4.4520. Min Act: -0.1700

Data Index: 8346130 (Open Web Text)

Max Activating Token Index: 671

Text #12

Max Range: 4.5583. Min Range: -4.5583

Max Act: 4.4256. Min Act: -0.1700

Data Index: 2712812 (Open Web Text)

Max Activating Token Index: 16

Text #13

Max Range: 4.5583. Min Range: -4.5583

Max Act: 4.4423. Min Act: -0.1700

Data Index: 6116339 (Open Web Text)

Max Activating Token Index: 911

Text #14

Max Range: 4.5583. Min Range: -4.5583

Max Act: 4.4128. Min Act: -0.1700

Data Index: 8770376 (Open Web Text)

Max Activating Token Index: 292

Text #15

Max Range: 4.5583. Min Range: -4.5583

Max Act: 4.3789. Min Act: -0.1700

Data Index: 4798261 (Open Web Text)

Max Activating Token Index: 490

Text #16

Max Range: 4.5583. Min Range: -4.5583

Max Act: 4.3796. Min Act: -0.1700

Data Index: 8761331 (Open Web Text)

Max Activating Token Index: 55

Text #17

Max Range: 4.5583. Min Range: -4.5583

Max Act: 4.3969. Min Act: -0.1700

Data Index: 1327559 (Open Web Text)

Max Activating Token Index: 470

Text #18

Max Range: 4.5583. Min Range: -4.5583

Max Act: 4.3889. Min Act: -0.1700

Data Index: 3678053 (Open Web Text)

Max Activating Token Index: 76

Text #19

Max Range: 4.5583. Min Range: -4.5583

Max Act: 4.3702. Min Act: -0.1700

Data Index: 5823313 (Open Web Text)

Max Activating Token Index: 1020