Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 747 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.1929. Min Range: -3.1929

Max Act: 3.1929. Min Act: -0.1700

Data Index: 1753747 (Open Web Text)

Max Activating Token Index: 4

Text #1

Max Range: 3.1929. Min Range: -3.1929

Max Act: 3.1735. Min Act: -0.1700

Data Index: 1176757 (Open Web Text)

Max Activating Token Index: 2

Text #2

Max Range: 3.1929. Min Range: -3.1929

Max Act: 3.1761. Min Act: -0.1700

Data Index: 7555205 (Open Web Text)

Max Activating Token Index: 4

Text #3

Max Range: 3.1929. Min Range: -3.1929

Max Act: 3.1655. Min Act: -0.1700

Data Index: 6391851 (Open Web Text)

Max Activating Token Index: 5

Text #4

Max Range: 3.1929. Min Range: -3.1929

Max Act: 3.1063. Min Act: -0.1700

Data Index: 7163136 (Open Web Text)

Max Activating Token Index: 2

Text #5

Max Range: 3.1929. Min Range: -3.1929

Max Act: 3.0805. Min Act: -0.1700

Data Index: 2755215 (Open Web Text)

Max Activating Token Index: 5

Text #6

Max Range: 3.1929. Min Range: -3.1929

Max Act: 3.0763. Min Act: -0.1700

Data Index: 3277988 (Open Web Text)

Max Activating Token Index: 3

Text #7

Max Range: 3.1929. Min Range: -3.1929

Max Act: 3.0517. Min Act: -0.1700

Data Index: 7147808 (Open Web Text)

Max Activating Token Index: 12

Text #8

Max Range: 3.1929. Min Range: -3.1929

Max Act: 3.0623. Min Act: -0.1700

Data Index: 7407737 (Open Web Text)

Max Activating Token Index: 6

Text #9

Max Range: 3.1929. Min Range: -3.1929

Max Act: 3.0518. Min Act: -0.1700

Data Index: 7128581 (Open Web Text)

Max Activating Token Index: 9

Text #10

Max Range: 3.1929. Min Range: -3.1929

Max Act: 3.0320. Min Act: -0.1700

Data Index: 403011 (Open Web Text)

Max Activating Token Index: 10

Text #11

Max Range: 3.1929. Min Range: -3.1929

Max Act: 3.0332. Min Act: -0.1700

Data Index: 6298392 (Open Web Text)

Max Activating Token Index: 4

Text #12

Max Range: 3.1929. Min Range: -3.1929

Max Act: 3.0203. Min Act: -0.1700

Data Index: 585074 (Open Web Text)

Max Activating Token Index: 3

Text #13

Max Range: 3.1929. Min Range: -3.1929

Max Act: 3.0230. Min Act: -0.1700

Data Index: 516465 (Open Web Text)

Max Activating Token Index: 4

Text #14

Max Range: 3.1929. Min Range: -3.1929

Max Act: 3.0133. Min Act: -0.1700

Data Index: 6961325 (Open Web Text)

Max Activating Token Index: 10

Text #15

Max Range: 3.1929. Min Range: -3.1929

Max Act: 3.0148. Min Act: -0.1700

Data Index: 7029327 (Open Web Text)

Max Activating Token Index: 3

Text #16

Max Range: 3.1929. Min Range: -3.1929

Max Act: 3.0226. Min Act: -0.1700

Data Index: 8746704 (Open Web Text)

Max Activating Token Index: 4

Text #17

Max Range: 3.1929. Min Range: -3.1929

Max Act: 3.0155. Min Act: -0.1700

Data Index: 3407067 (Open Web Text)

Max Activating Token Index: 5

Text #18

Max Range: 3.1929. Min Range: -3.1929

Max Act: 3.0089. Min Act: -0.1700

Data Index: 6185645 (Open Web Text)

Max Activating Token Index: 5

Text #19

Max Range: 3.1929. Min Range: -3.1929

Max Act: 2.9896. Min Act: -0.1700

Data Index: 3547071 (Open Web Text)

Max Activating Token Index: 5