Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1023 in Layer 3

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.0759. Min Range: -3.0759

Max Act: 3.0532. Min Act: -0.1700

Data Index: 2696694 (Open Web Text)

Max Activating Token Index: 1023

Text #1

Max Range: 3.0759. Min Range: -3.0759

Max Act: 3.0759. Min Act: -0.1700

Data Index: 6479700 (Open Web Text)

Max Activating Token Index: 109

Text #2

Max Range: 3.0759. Min Range: -3.0759

Max Act: 3.0494. Min Act: -0.1700

Data Index: 3862874 (Open Web Text)

Max Activating Token Index: 687

Text #3

Max Range: 3.0759. Min Range: -3.0759

Max Act: 3.0380. Min Act: -0.1700

Data Index: 4559149 (Open Web Text)

Max Activating Token Index: 744

Text #4

Max Range: 3.0759. Min Range: -3.0759

Max Act: 3.0290. Min Act: -0.1700

Data Index: 4216241 (Open Web Text)

Max Activating Token Index: 238

Text #5

Max Range: 3.0759. Min Range: -3.0759

Max Act: 3.0003. Min Act: -0.1700

Data Index: 2374249 (Open Web Text)

Max Activating Token Index: 747

Text #6

Max Range: 3.0759. Min Range: -3.0759

Max Act: 2.9798. Min Act: -0.1700

Data Index: 8729096 (Open Web Text)

Max Activating Token Index: 719

Text #7

Max Range: 3.0759. Min Range: -3.0759

Max Act: 3.0188. Min Act: -0.1700

Data Index: 3967092 (Open Web Text)

Max Activating Token Index: 766

Text #8

Max Range: 3.0759. Min Range: -3.0759

Max Act: 3.0154. Min Act: -0.1700

Data Index: 7795715 (Open Web Text)

Max Activating Token Index: 320

Text #9

Max Range: 3.0759. Min Range: -3.0759

Max Act: 3.0021. Min Act: -0.1700

Data Index: 6632289 (Open Web Text)

Max Activating Token Index: 576

Text #10

Max Range: 3.0759. Min Range: -3.0759

Max Act: 2.9835. Min Act: -0.1700

Data Index: 8348184 (Open Web Text)

Max Activating Token Index: 226

Text #11

Max Range: 3.0759. Min Range: -3.0759

Max Act: 2.9981. Min Act: -0.1700

Data Index: 2366362 (Open Web Text)

Max Activating Token Index: 591

Text #12

Max Range: 3.0759. Min Range: -3.0759

Max Act: 3.0135. Min Act: -0.1700

Data Index: 5057830 (Open Web Text)

Max Activating Token Index: 903

Text #13

Max Range: 3.0759. Min Range: -3.0759

Max Act: 2.9782. Min Act: -0.1700

Data Index: 7071021 (Open Web Text)

Max Activating Token Index: 1013

Text #14

Max Range: 3.0759. Min Range: -3.0759

Max Act: 2.9675. Min Act: -0.1700

Data Index: 760093 (Open Web Text)

Max Activating Token Index: 531

Text #15

Max Range: 3.0759. Min Range: -3.0759

Max Act: 2.9852. Min Act: -0.1700

Data Index: 799396 (Open Web Text)

Max Activating Token Index: 998

Text #16

Max Range: 3.0759. Min Range: -3.0759

Max Act: 2.9670. Min Act: -0.1700

Data Index: 5917334 (Open Web Text)

Max Activating Token Index: 223

Text #17

Max Range: 3.0759. Min Range: -3.0759

Max Act: 2.9721. Min Act: -0.1700

Data Index: 2708610 (Open Web Text)

Max Activating Token Index: 714

Text #18

Max Range: 3.0759. Min Range: -3.0759

Max Act: 2.9763. Min Act: -0.1700

Data Index: 7487503 (Open Web Text)

Max Activating Token Index: 236

Text #19

Max Range: 3.0759. Min Range: -3.0759

Max Act: 2.9603. Min Act: -0.1700

Data Index: 7696402 (Open Web Text)

Max Activating Token Index: 266