Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1340 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.2846. Min Range: -4.2846

Max Act: 4.2846. Min Act: -0.1700

Data Index: 3025397 (Open Web Text)

Max Activating Token Index: 4

Text #1

Max Range: 4.2846. Min Range: -4.2846

Max Act: 4.1360. Min Act: -0.1700

Data Index: 5116247 (Open Web Text)

Max Activating Token Index: 4

Text #2

Max Range: 4.2846. Min Range: -4.2846

Max Act: 4.1484. Min Act: -0.1700

Data Index: 1961252 (Open Web Text)

Max Activating Token Index: 6

Text #3

Max Range: 4.2846. Min Range: -4.2846

Max Act: 4.1000. Min Act: -0.1700

Data Index: 1290922 (Open Web Text)

Max Activating Token Index: 4

Text #4

Max Range: 4.2846. Min Range: -4.2846

Max Act: 4.0661. Min Act: -0.1700

Data Index: 2579199 (Open Web Text)

Max Activating Token Index: 5

Text #5

Max Range: 4.2846. Min Range: -4.2846

Max Act: 4.0791. Min Act: -0.1700

Data Index: 5186756 (Open Web Text)

Max Activating Token Index: 4

Text #6

Max Range: 4.2846. Min Range: -4.2846

Max Act: 4.0860. Min Act: -0.1700

Data Index: 6400948 (Open Web Text)

Max Activating Token Index: 4

Text #7

Max Range: 4.2846. Min Range: -4.2846

Max Act: 4.0843. Min Act: -0.1700

Data Index: 3382445 (Open Web Text)

Max Activating Token Index: 5

Text #8

Max Range: 4.2846. Min Range: -4.2846

Max Act: 4.0630. Min Act: -0.1700

Data Index: 6509598 (Open Web Text)

Max Activating Token Index: 5

Text #9

Max Range: 4.2846. Min Range: -4.2846

Max Act: 4.0729. Min Act: -0.1700

Data Index: 1214837 (Open Web Text)

Max Activating Token Index: 11

Text #10

Max Range: 4.2846. Min Range: -4.2846

Max Act: 4.0949. Min Act: -0.1700

Data Index: 897090 (Open Web Text)

Max Activating Token Index: 5

Text #11

Max Range: 4.2846. Min Range: -4.2846

Max Act: 4.0694. Min Act: -0.1700

Data Index: 231212 (Open Web Text)

Max Activating Token Index: 11

Text #12

Max Range: 4.2846. Min Range: -4.2846

Max Act: 4.0521. Min Act: -0.1700

Data Index: 2341337 (Open Web Text)

Max Activating Token Index: 7

Text #13

Max Range: 4.2846. Min Range: -4.2846

Max Act: 4.0563. Min Act: -0.1700

Data Index: 4462213 (Open Web Text)

Max Activating Token Index: 7

Text #14

Max Range: 4.2846. Min Range: -4.2846

Max Act: 4.0601. Min Act: -0.1700

Data Index: 1469423 (Open Web Text)

Max Activating Token Index: 6

Text #15

Max Range: 4.2846. Min Range: -4.2846

Max Act: 4.0543. Min Act: -0.1700

Data Index: 1084467 (Open Web Text)

Max Activating Token Index: 31

Text #16

Max Range: 4.2846. Min Range: -4.2846

Max Act: 4.0486. Min Act: -0.1700

Data Index: 1274107 (Open Web Text)

Max Activating Token Index: 15

Text #17

Max Range: 4.2846. Min Range: -4.2846

Max Act: 4.0636. Min Act: -0.1700

Data Index: 4445799 (Open Web Text)

Max Activating Token Index: 6

Text #18

Max Range: 4.2846. Min Range: -4.2846

Max Act: 4.0471. Min Act: -0.1700

Data Index: 4369994 (Open Web Text)

Max Activating Token Index: 6

Text #19

Max Range: 4.2846. Min Range: -4.2846

Max Act: 4.0372. Min Act: -0.1700

Data Index: 3985874 (Open Web Text)

Max Activating Token Index: 8