Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2358 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.8766. Min Range: -4.8766

Max Act: 4.8766. Min Act: -0.1700

Data Index: 1560530 (Open Web Text)

Max Activating Token Index: 842

Text #1

Max Range: 4.8766. Min Range: -4.8766

Max Act: 4.7784. Min Act: -0.1700

Data Index: 2287813 (Open Web Text)

Max Activating Token Index: 456

Text #2

Max Range: 4.8766. Min Range: -4.8766

Max Act: 4.6567. Min Act: -0.1700

Data Index: 5633419 (Open Web Text)

Max Activating Token Index: 424

Text #3

Max Range: 4.8766. Min Range: -4.8766

Max Act: 4.6796. Min Act: -0.1700

Data Index: 2476424 (Open Web Text)

Max Activating Token Index: 483

Text #4

Max Range: 4.8766. Min Range: -4.8766

Max Act: 4.6223. Min Act: -0.1700

Data Index: 7043981 (Open Web Text)

Max Activating Token Index: 506

Text #5

Max Range: 4.8766. Min Range: -4.8766

Max Act: 4.6618. Min Act: -0.1700

Data Index: 8450612 (Open Web Text)

Max Activating Token Index: 314

Text #6

Max Range: 4.8766. Min Range: -4.8766

Max Act: 4.6707. Min Act: -0.1700

Data Index: 3891119 (Open Web Text)

Max Activating Token Index: 588

Text #7

Max Range: 4.8766. Min Range: -4.8766

Max Act: 4.6639. Min Act: -0.1700

Data Index: 4659151 (Open Web Text)

Max Activating Token Index: 671

Text #8

Max Range: 4.8766. Min Range: -4.8766

Max Act: 4.6017. Min Act: -0.1700

Data Index: 6433711 (Open Web Text)

Max Activating Token Index: 609

Text #9

Max Range: 4.8766. Min Range: -4.8766

Max Act: 4.6404. Min Act: -0.1700

Data Index: 7985130 (Open Web Text)

Max Activating Token Index: 820

Text #10

Max Range: 4.8766. Min Range: -4.8766

Max Act: 4.6251. Min Act: -0.1700

Data Index: 2602647 (Open Web Text)

Max Activating Token Index: 988

Text #11

Max Range: 4.8766. Min Range: -4.8766

Max Act: 4.6270. Min Act: -0.1700

Data Index: 336585 (Open Web Text)

Max Activating Token Index: 750

Text #12

Max Range: 4.8766. Min Range: -4.8766

Max Act: 4.6072. Min Act: -0.1700

Data Index: 7017314 (Open Web Text)

Max Activating Token Index: 789

Text #13

Max Range: 4.8766. Min Range: -4.8766

Max Act: 4.6288. Min Act: -0.1700

Data Index: 7966920 (Open Web Text)

Max Activating Token Index: 150

Text #14

Max Range: 4.8766. Min Range: -4.8766

Max Act: 4.5920. Min Act: -0.1700

Data Index: 2909848 (Open Web Text)

Max Activating Token Index: 317

Text #15

Max Range: 4.8766. Min Range: -4.8766

Max Act: 4.6017. Min Act: -0.1700

Data Index: 1655372 (Open Web Text)

Max Activating Token Index: 714

Text #16

Max Range: 4.8766. Min Range: -4.8766

Max Act: 4.5893. Min Act: -0.1700

Data Index: 3516275 (Open Web Text)

Max Activating Token Index: 531

Text #17

Max Range: 4.8766. Min Range: -4.8766

Max Act: 4.6205. Min Act: -0.1700

Data Index: 6558371 (Open Web Text)

Max Activating Token Index: 148

Text #18

Max Range: 4.8766. Min Range: -4.8766

Max Act: 4.5815. Min Act: -0.1700

Data Index: 8610851 (Open Web Text)

Max Activating Token Index: 82

Text #19

Max Range: 4.8766. Min Range: -4.8766

Max Act: 4.5603. Min Act: -0.1700

Data Index: 6700133 (Open Web Text)

Max Activating Token Index: 245