Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 616 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.3409. Min Range: -4.3409

Max Act: 4.3409. Min Act: -0.1700

Data Index: 7492367 (Open Web Text)

Max Activating Token Index: 4

Text #1

Max Range: 4.3409. Min Range: -4.3409

Max Act: 4.2798. Min Act: -0.1700

Data Index: 7713574 (Open Web Text)

Max Activating Token Index: 13

Text #2

Max Range: 4.3409. Min Range: -4.3409

Max Act: 4.1932. Min Act: -0.1700

Data Index: 4857360 (Open Web Text)

Max Activating Token Index: 16

Text #3

Max Range: 4.3409. Min Range: -4.3409

Max Act: 4.2023. Min Act: -0.1700

Data Index: 8713294 (Open Web Text)

Max Activating Token Index: 3

Text #4

Max Range: 4.3409. Min Range: -4.3409

Max Act: 4.2006. Min Act: -0.1700

Data Index: 5041656 (Open Web Text)

Max Activating Token Index: 18

Text #5

Max Range: 4.3409. Min Range: -4.3409

Max Act: 4.1706. Min Act: -0.1700

Data Index: 6607596 (Open Web Text)

Max Activating Token Index: 3

Text #6

Max Range: 4.3409. Min Range: -4.3409

Max Act: 4.1878. Min Act: -0.1700

Data Index: 8225615 (Open Web Text)

Max Activating Token Index: 19

Text #7

Max Range: 4.3409. Min Range: -4.3409

Max Act: 4.1597. Min Act: -0.1700

Data Index: 8707614 (Open Web Text)

Max Activating Token Index: 32

Text #8

Max Range: 4.3409. Min Range: -4.3409

Max Act: 4.1861. Min Act: -0.1700

Data Index: 37538 (Open Web Text)

Max Activating Token Index: 6

Text #9

Max Range: 4.3409. Min Range: -4.3409

Max Act: 4.1635. Min Act: -0.1700

Data Index: 5322465 (Open Web Text)

Max Activating Token Index: 8

Text #10

Max Range: 4.3409. Min Range: -4.3409

Max Act: 4.1732. Min Act: -0.1700

Data Index: 5427835 (Open Web Text)

Max Activating Token Index: 12

Text #11

Max Range: 4.3409. Min Range: -4.3409

Max Act: 4.1458. Min Act: -0.1700

Data Index: 2267437 (Open Web Text)

Max Activating Token Index: 32

Text #12

Max Range: 4.3409. Min Range: -4.3409

Max Act: 4.1472. Min Act: -0.1700

Data Index: 2381593 (Open Web Text)

Max Activating Token Index: 10

Text #13

Max Range: 4.3409. Min Range: -4.3409

Max Act: 4.1252. Min Act: -0.1700

Data Index: 3793076 (Open Web Text)

Max Activating Token Index: 4

Text #14

Max Range: 4.3409. Min Range: -4.3409

Max Act: 4.1217. Min Act: -0.1700

Data Index: 5707303 (Open Web Text)

Max Activating Token Index: 41

Text #15

Max Range: 4.3409. Min Range: -4.3409

Max Act: 4.1248. Min Act: -0.1700

Data Index: 4817434 (Open Web Text)

Max Activating Token Index: 4

Text #16

Max Range: 4.3409. Min Range: -4.3409

Max Act: 4.1059. Min Act: -0.1700

Data Index: 7578983 (Open Web Text)

Max Activating Token Index: 34

Text #17

Max Range: 4.3409. Min Range: -4.3409

Max Act: 4.0931. Min Act: -0.1700

Data Index: 4766558 (Open Web Text)

Max Activating Token Index: 16

Text #18

Max Range: 4.3409. Min Range: -4.3409

Max Act: 4.0907. Min Act: -0.1700

Data Index: 1309502 (Open Web Text)

Max Activating Token Index: 4

Text #19

Max Range: 4.3409. Min Range: -4.3409

Max Act: 4.0939. Min Act: -0.1700

Data Index: 7492366 (Open Web Text)

Max Activating Token Index: 863