Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1766 in Layer 2

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.8330. Min Range: -3.8330

Max Act: 3.8330. Min Act: -0.1700

Data Index: 4788694 (Open Web Text)

Max Activating Token Index: 329

Text #1

Max Range: 3.8330. Min Range: -3.8330

Max Act: 3.7696. Min Act: -0.1700

Data Index: 5163610 (Open Web Text)

Max Activating Token Index: 852

Text #2

Max Range: 3.8330. Min Range: -3.8330

Max Act: 3.7630. Min Act: -0.1700

Data Index: 1245616 (Open Web Text)

Max Activating Token Index: 493

Text #3

Max Range: 3.8330. Min Range: -3.8330

Max Act: 3.7486. Min Act: -0.1700

Data Index: 239861 (Open Web Text)

Max Activating Token Index: 280

Text #4

Max Range: 3.8330. Min Range: -3.8330

Max Act: 3.7485. Min Act: -0.1700

Data Index: 3235979 (Open Web Text)

Max Activating Token Index: 193

Text #5

Max Range: 3.8330. Min Range: -3.8330

Max Act: 3.7239. Min Act: -0.1700

Data Index: 5186657 (Open Web Text)

Max Activating Token Index: 41

Text #6

Max Range: 3.8330. Min Range: -3.8330

Max Act: 3.6967. Min Act: -0.1700

Data Index: 5561029 (Open Web Text)

Max Activating Token Index: 462

Text #7

Max Range: 3.8330. Min Range: -3.8330

Max Act: 3.6639. Min Act: -0.1700

Data Index: 3227556 (Open Web Text)

Max Activating Token Index: 769

Text #8

Max Range: 3.8330. Min Range: -3.8330

Max Act: 3.7008. Min Act: -0.1700

Data Index: 7958421 (Open Web Text)

Max Activating Token Index: 741

Text #9

Max Range: 3.8330. Min Range: -3.8330

Max Act: 3.6895. Min Act: -0.1700

Data Index: 3127862 (Open Web Text)

Max Activating Token Index: 835

Text #10

Max Range: 3.8330. Min Range: -3.8330

Max Act: 3.6766. Min Act: -0.1700

Data Index: 7853398 (Open Web Text)

Max Activating Token Index: 422

Text #11

Max Range: 3.8330. Min Range: -3.8330

Max Act: 3.6859. Min Act: -0.1700

Data Index: 3125907 (Open Web Text)

Max Activating Token Index: 841

Text #12

Max Range: 3.8330. Min Range: -3.8330

Max Act: 3.7074. Min Act: -0.1700

Data Index: 1095942 (Open Web Text)

Max Activating Token Index: 835

Text #13

Max Range: 3.8330. Min Range: -3.8330

Max Act: 3.6827. Min Act: -0.1700

Data Index: 6436544 (Open Web Text)

Max Activating Token Index: 527

Text #14

Max Range: 3.8330. Min Range: -3.8330

Max Act: 3.7046. Min Act: -0.1700

Data Index: 5258320 (Open Web Text)

Max Activating Token Index: 114

Text #15

Max Range: 3.8330. Min Range: -3.8330

Max Act: 3.6993. Min Act: -0.1700

Data Index: 4824405 (Open Web Text)

Max Activating Token Index: 861

Text #16

Max Range: 3.8330. Min Range: -3.8330

Max Act: 3.7154. Min Act: -0.1700

Data Index: 2084871 (Open Web Text)

Max Activating Token Index: 204

Text #17

Max Range: 3.8330. Min Range: -3.8330

Max Act: 3.6744. Min Act: -0.1700

Data Index: 2021879 (Open Web Text)

Max Activating Token Index: 361

Text #18

Max Range: 3.8330. Min Range: -3.8330

Max Act: 3.6926. Min Act: -0.1700

Data Index: 4001879 (Open Web Text)

Max Activating Token Index: 487

Text #19

Max Range: 3.8330. Min Range: -3.8330

Max Act: 3.7002. Min Act: -0.1700

Data Index: 3492314 (Open Web Text)

Max Activating Token Index: 557