Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1066 in Layer 2

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 2.8573. Min Range: -2.8573

Max Act: 2.8573. Min Act: -0.1700

Data Index: 3959404 (Open Web Text)

Max Activating Token Index: 1023

Text #1

Max Range: 2.8573. Min Range: -2.8573

Max Act: 2.7900. Min Act: -0.1700

Data Index: 7360421 (Open Web Text)

Max Activating Token Index: 1023

Text #2

Max Range: 2.8573. Min Range: -2.8573

Max Act: 2.7787. Min Act: -0.1700

Data Index: 8694463 (Open Web Text)

Max Activating Token Index: 1023

Text #3

Max Range: 2.8573. Min Range: -2.8573

Max Act: 2.7492. Min Act: -0.1700

Data Index: 4540160 (Open Web Text)

Max Activating Token Index: 1023

Text #4

Max Range: 2.8573. Min Range: -2.8573

Max Act: 2.7594. Min Act: -0.1700

Data Index: 7822736 (Open Web Text)

Max Activating Token Index: 1023

Text #5

Max Range: 2.8573. Min Range: -2.8573

Max Act: 2.7199. Min Act: -0.1700

Data Index: 7966874 (Open Web Text)

Max Activating Token Index: 1023

Text #6

Max Range: 2.8573. Min Range: -2.8573

Max Act: 2.7012. Min Act: -0.1700

Data Index: 2000819 (Open Web Text)

Max Activating Token Index: 1023

Text #7

Max Range: 2.8573. Min Range: -2.8573

Max Act: 2.7075. Min Act: -0.1700

Data Index: 7983511 (Open Web Text)

Max Activating Token Index: 1023

Text #8

Max Range: 2.8573. Min Range: -2.8573

Max Act: 2.6726. Min Act: -0.1700

Data Index: 5522314 (Open Web Text)

Max Activating Token Index: 1023

Text #9

Max Range: 2.8573. Min Range: -2.8573

Max Act: 2.6933. Min Act: -0.1700

Data Index: 1132144 (Open Web Text)

Max Activating Token Index: 1023

Text #10

Max Range: 2.8573. Min Range: -2.8573

Max Act: 2.6788. Min Act: -0.1700

Data Index: 641345 (Open Web Text)

Max Activating Token Index: 1023

Text #11

Max Range: 2.8573. Min Range: -2.8573

Max Act: 2.6661. Min Act: -0.1700

Data Index: 7418718 (Open Web Text)

Max Activating Token Index: 1023

Text #12

Max Range: 2.8573. Min Range: -2.8573

Max Act: 2.6744. Min Act: -0.1700

Data Index: 4474567 (Open Web Text)

Max Activating Token Index: 1023

Text #13

Max Range: 2.8573. Min Range: -2.8573

Max Act: 2.6499. Min Act: -0.1700

Data Index: 1741576 (Open Web Text)

Max Activating Token Index: 1023

Text #14

Max Range: 2.8573. Min Range: -2.8573

Max Act: 2.6507. Min Act: -0.1700

Data Index: 2055273 (Open Web Text)

Max Activating Token Index: 1023

Text #15

Max Range: 2.8573. Min Range: -2.8573

Max Act: 2.6460. Min Act: -0.1700

Data Index: 8065012 (Open Web Text)

Max Activating Token Index: 1023

Text #16

Max Range: 2.8573. Min Range: -2.8573

Max Act: 2.6445. Min Act: -0.1700

Data Index: 328164 (Open Web Text)

Max Activating Token Index: 1023

Text #17

Max Range: 2.8573. Min Range: -2.8573

Max Act: 2.6336. Min Act: -0.1700

Data Index: 5124605 (Open Web Text)

Max Activating Token Index: 1023

Text #18

Max Range: 2.8573. Min Range: -2.8573

Max Act: 2.6285. Min Act: -0.1700

Data Index: 3508747 (Open Web Text)

Max Activating Token Index: 1023

Text #19

Max Range: 2.8573. Min Range: -2.8573

Max Act: 2.6272. Min Act: -0.1700

Data Index: 709611 (Open Web Text)

Max Activating Token Index: 1023