Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1677 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 6.0092. Min Range: -6.0092

Max Act: 6.0092. Min Act: -0.1700

Data Index: 6732490 (Open Web Text)

Max Activating Token Index: 411

Text #1

Max Range: 6.0092. Min Range: -6.0092

Max Act: 5.8671. Min Act: -0.1700

Data Index: 6875766 (Open Web Text)

Max Activating Token Index: 186

Text #2

Max Range: 6.0092. Min Range: -6.0092

Max Act: 5.7514. Min Act: -0.1700

Data Index: 5043045 (Open Web Text)

Max Activating Token Index: 268

Text #3

Max Range: 6.0092. Min Range: -6.0092

Max Act: 5.7935. Min Act: -0.1700

Data Index: 7218217 (Open Web Text)

Max Activating Token Index: 965

Text #4

Max Range: 6.0092. Min Range: -6.0092

Max Act: 5.7064. Min Act: -0.1700

Data Index: 6204498 (Open Web Text)

Max Activating Token Index: 468

Text #5

Max Range: 6.0092. Min Range: -6.0092

Max Act: 5.6120. Min Act: -0.1700

Data Index: 8573246 (Open Web Text)

Max Activating Token Index: 1012

Text #6

Max Range: 6.0092. Min Range: -6.0092

Max Act: 5.5347. Min Act: -0.1700

Data Index: 4666504 (Open Web Text)

Max Activating Token Index: 710

Text #7

Max Range: 6.0092. Min Range: -6.0092

Max Act: 5.4861. Min Act: -0.1700

Data Index: 5645755 (Open Web Text)

Max Activating Token Index: 244

Text #8

Max Range: 6.0092. Min Range: -6.0092

Max Act: 5.4853. Min Act: -0.1700

Data Index: 2459595 (Open Web Text)

Max Activating Token Index: 876

Text #9

Max Range: 6.0092. Min Range: -6.0092

Max Act: 5.4606. Min Act: -0.1700

Data Index: 928249 (Open Web Text)

Max Activating Token Index: 277

Text #10

Max Range: 6.0092. Min Range: -6.0092

Max Act: 5.4884. Min Act: -0.1700

Data Index: 1408649 (Open Web Text)

Max Activating Token Index: 487

Text #11

Max Range: 6.0092. Min Range: -6.0092

Max Act: 5.5193. Min Act: -0.1700

Data Index: 3519574 (Open Web Text)

Max Activating Token Index: 336

Text #12

Max Range: 6.0092. Min Range: -6.0092

Max Act: 5.5036. Min Act: -0.1700

Data Index: 3959850 (Open Web Text)

Max Activating Token Index: 450

Text #13

Max Range: 6.0092. Min Range: -6.0092

Max Act: 5.5201. Min Act: -0.1700

Data Index: 2518550 (Open Web Text)

Max Activating Token Index: 105

Text #14

Max Range: 6.0092. Min Range: -6.0092

Max Act: 5.4440. Min Act: -0.1700

Data Index: 7586409 (Open Web Text)

Max Activating Token Index: 845

Text #15

Max Range: 6.0092. Min Range: -6.0092

Max Act: 5.4603. Min Act: -0.1700

Data Index: 3910862 (Open Web Text)

Max Activating Token Index: 1016

Text #16

Max Range: 6.0092. Min Range: -6.0092

Max Act: 5.4538. Min Act: -0.1700

Data Index: 8066070 (Open Web Text)

Max Activating Token Index: 186

Text #17

Max Range: 6.0092. Min Range: -6.0092

Max Act: 5.4170. Min Act: -0.1700

Data Index: 341856 (Open Web Text)

Max Activating Token Index: 296

Text #18

Max Range: 6.0092. Min Range: -6.0092

Max Act: 5.4263. Min Act: -0.1700

Data Index: 5635411 (Open Web Text)

Max Activating Token Index: 753

Text #19

Max Range: 6.0092. Min Range: -6.0092

Max Act: 5.4048. Min Act: -0.1700

Data Index: 830490 (Open Web Text)

Max Activating Token Index: 982