Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1733 in Layer 2

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.4985. Min Range: -3.4985

Max Act: 3.4985. Min Act: -0.1700

Data Index: 5517786 (Open Web Text)

Max Activating Token Index: 936

Text #1

Max Range: 3.4985. Min Range: -3.4985

Max Act: 3.4531. Min Act: -0.1700

Data Index: 2528940 (Open Web Text)

Max Activating Token Index: 213

Text #2

Max Range: 3.4985. Min Range: -3.4985

Max Act: 3.4134. Min Act: -0.1700

Data Index: 6010766 (Open Web Text)

Max Activating Token Index: 643

Text #3

Max Range: 3.4985. Min Range: -3.4985

Max Act: 3.4051. Min Act: -0.1700

Data Index: 2953884 (Open Web Text)

Max Activating Token Index: 44

Text #4

Max Range: 3.4985. Min Range: -3.4985

Max Act: 3.4143. Min Act: -0.1700

Data Index: 4034339 (Open Web Text)

Max Activating Token Index: 947

Text #5

Max Range: 3.4985. Min Range: -3.4985

Max Act: 3.4077. Min Act: -0.1700

Data Index: 3139237 (Open Web Text)

Max Activating Token Index: 760

Text #6

Max Range: 3.4985. Min Range: -3.4985

Max Act: 3.3849. Min Act: -0.1700

Data Index: 2685955 (Open Web Text)

Max Activating Token Index: 1014

Text #7

Max Range: 3.4985. Min Range: -3.4985

Max Act: 3.4130. Min Act: -0.1700

Data Index: 8165251 (Open Web Text)

Max Activating Token Index: 1011

Text #8

Max Range: 3.4985. Min Range: -3.4985

Max Act: 3.3595. Min Act: -0.1700

Data Index: 2259749 (Open Web Text)

Max Activating Token Index: 685

Text #9

Max Range: 3.4985. Min Range: -3.4985

Max Act: 3.3399. Min Act: -0.1700

Data Index: 4501442 (Open Web Text)

Max Activating Token Index: 532

Text #10

Max Range: 3.4985. Min Range: -3.4985

Max Act: 3.3763. Min Act: -0.1700

Data Index: 2113532 (Open Web Text)

Max Activating Token Index: 151

Text #11

Max Range: 3.4985. Min Range: -3.4985

Max Act: 3.3588. Min Act: -0.1700

Data Index: 1468418 (Open Web Text)

Max Activating Token Index: 624

Text #12

Max Range: 3.4985. Min Range: -3.4985

Max Act: 3.3751. Min Act: -0.1700

Data Index: 5018522 (Open Web Text)

Max Activating Token Index: 415

Text #13

Max Range: 3.4985. Min Range: -3.4985

Max Act: 3.3465. Min Act: -0.1700

Data Index: 8270096 (Open Web Text)

Max Activating Token Index: 888

Text #14

Max Range: 3.4985. Min Range: -3.4985

Max Act: 3.3297. Min Act: -0.1700

Data Index: 2156851 (Open Web Text)

Max Activating Token Index: 700

Text #15

Max Range: 3.4985. Min Range: -3.4985

Max Act: 3.3349. Min Act: -0.1700

Data Index: 4799827 (Open Web Text)

Max Activating Token Index: 155

Text #16

Max Range: 3.4985. Min Range: -3.4985

Max Act: 3.3459. Min Act: -0.1700

Data Index: 4827430 (Open Web Text)

Max Activating Token Index: 724

Text #17

Max Range: 3.4985. Min Range: -3.4985

Max Act: 3.3280. Min Act: -0.1700

Data Index: 662166 (Open Web Text)

Max Activating Token Index: 660

Text #18

Max Range: 3.4985. Min Range: -3.4985

Max Act: 3.3073. Min Act: -0.1700

Data Index: 1063417 (Open Web Text)

Max Activating Token Index: 678

Text #19

Max Range: 3.4985. Min Range: -3.4985

Max Act: 3.3179. Min Act: -0.1700

Data Index: 4490221 (Open Web Text)

Max Activating Token Index: 610