Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 3046 in Layer 3

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.2687. Min Range: -4.2687

Max Act: 4.2515. Min Act: -0.1700

Data Index: 7590400 (Open Web Text)

Max Activating Token Index: 915

Text #1

Max Range: 4.2687. Min Range: -4.2687

Max Act: 4.2400. Min Act: -0.1700

Data Index: 2498252 (Open Web Text)

Max Activating Token Index: 876

Text #2

Max Range: 4.2687. Min Range: -4.2687

Max Act: 4.2687. Min Act: -0.1700

Data Index: 6289604 (Open Web Text)

Max Activating Token Index: 780

Text #3

Max Range: 4.2687. Min Range: -4.2687

Max Act: 4.2248. Min Act: -0.1700

Data Index: 5643945 (Open Web Text)

Max Activating Token Index: 386

Text #4

Max Range: 4.2687. Min Range: -4.2687

Max Act: 4.2212. Min Act: -0.1700

Data Index: 2146831 (Open Web Text)

Max Activating Token Index: 191

Text #5

Max Range: 4.2687. Min Range: -4.2687

Max Act: 4.2115. Min Act: -0.1700

Data Index: 7546694 (Open Web Text)

Max Activating Token Index: 916

Text #6

Max Range: 4.2687. Min Range: -4.2687

Max Act: 4.2219. Min Act: -0.1700

Data Index: 3444890 (Open Web Text)

Max Activating Token Index: 709

Text #7

Max Range: 4.2687. Min Range: -4.2687

Max Act: 4.1986. Min Act: -0.1700

Data Index: 484719 (Open Web Text)

Max Activating Token Index: 664

Text #8

Max Range: 4.2687. Min Range: -4.2687

Max Act: 4.1961. Min Act: -0.1700

Data Index: 7277756 (Open Web Text)

Max Activating Token Index: 516

Text #9

Max Range: 4.2687. Min Range: -4.2687

Max Act: 4.1930. Min Act: -0.1700

Data Index: 7051263 (Open Web Text)

Max Activating Token Index: 615

Text #10

Max Range: 4.2687. Min Range: -4.2687

Max Act: 4.2089. Min Act: -0.1700

Data Index: 734535 (Open Web Text)

Max Activating Token Index: 998

Text #11

Max Range: 4.2687. Min Range: -4.2687

Max Act: 4.1468. Min Act: -0.1700

Data Index: 4523944 (Open Web Text)

Max Activating Token Index: 175

Text #12

Max Range: 4.2687. Min Range: -4.2687

Max Act: 4.1788. Min Act: -0.1700

Data Index: 795956 (Open Web Text)

Max Activating Token Index: 969

Text #13

Max Range: 4.2687. Min Range: -4.2687

Max Act: 4.1489. Min Act: -0.1700

Data Index: 3124534 (Open Web Text)

Max Activating Token Index: 590

Text #14

Max Range: 4.2687. Min Range: -4.2687

Max Act: 4.1604. Min Act: -0.1700

Data Index: 1852640 (Open Web Text)

Max Activating Token Index: 174

Text #15

Max Range: 4.2687. Min Range: -4.2687

Max Act: 4.1761. Min Act: -0.1700

Data Index: 6941284 (Open Web Text)

Max Activating Token Index: 993

Text #16

Max Range: 4.2687. Min Range: -4.2687

Max Act: 4.1692. Min Act: -0.1700

Data Index: 6481902 (Open Web Text)

Max Activating Token Index: 396

Text #17

Max Range: 4.2687. Min Range: -4.2687

Max Act: 4.1783. Min Act: -0.1700

Data Index: 6370374 (Open Web Text)

Max Activating Token Index: 876

Text #18

Max Range: 4.2687. Min Range: -4.2687

Max Act: 4.1648. Min Act: -0.1700

Data Index: 1147534 (Open Web Text)

Max Activating Token Index: 177

Text #19

Max Range: 4.2687. Min Range: -4.2687

Max Act: 4.1541. Min Act: -0.1700

Data Index: 264550 (Open Web Text)

Max Activating Token Index: 752