Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 546 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.5484. Min Range: -3.5484

Max Act: 3.5484. Min Act: -0.1700

Data Index: 5309740 (Open Web Text)

Max Activating Token Index: 585

Text #1

Max Range: 3.5484. Min Range: -3.5484

Max Act: 3.4487. Min Act: -0.1700

Data Index: 3396250 (Open Web Text)

Max Activating Token Index: 319

Text #2

Max Range: 3.5484. Min Range: -3.5484

Max Act: 3.4280. Min Act: -0.1700

Data Index: 1625994 (Open Web Text)

Max Activating Token Index: 563

Text #3

Max Range: 3.5484. Min Range: -3.5484

Max Act: 3.4302. Min Act: -0.1700

Data Index: 1556605 (Open Web Text)

Max Activating Token Index: 321

Text #4

Max Range: 3.5484. Min Range: -3.5484

Max Act: 3.3834. Min Act: -0.1700

Data Index: 2546546 (Open Web Text)

Max Activating Token Index: 593

Text #5

Max Range: 3.5484. Min Range: -3.5484

Max Act: 3.2767. Min Act: -0.1700

Data Index: 7410188 (Open Web Text)

Max Activating Token Index: 607

Text #6

Max Range: 3.5484. Min Range: -3.5484

Max Act: 3.3044. Min Act: -0.1700

Data Index: 8181190 (Open Web Text)

Max Activating Token Index: 868

Text #7

Max Range: 3.5484. Min Range: -3.5484

Max Act: 3.3058. Min Act: -0.1700

Data Index: 6207232 (Open Web Text)

Max Activating Token Index: 687

Text #8

Max Range: 3.5484. Min Range: -3.5484

Max Act: 3.2987. Min Act: -0.1700

Data Index: 7005111 (Open Web Text)

Max Activating Token Index: 1014

Text #9

Max Range: 3.5484. Min Range: -3.5484

Max Act: 3.2805. Min Act: -0.1700

Data Index: 3317636 (Open Web Text)

Max Activating Token Index: 1005

Text #10

Max Range: 3.5484. Min Range: -3.5484

Max Act: 3.2951. Min Act: -0.1700

Data Index: 2922636 (Open Web Text)

Max Activating Token Index: 816

Text #11

Max Range: 3.5484. Min Range: -3.5484

Max Act: 3.2517. Min Act: -0.1700

Data Index: 6170417 (Open Web Text)

Max Activating Token Index: 902

Text #12

Max Range: 3.5484. Min Range: -3.5484

Max Act: 3.2740. Min Act: -0.1700

Data Index: 4942877 (Open Web Text)

Max Activating Token Index: 189

Text #13

Max Range: 3.5484. Min Range: -3.5484

Max Act: 3.2513. Min Act: -0.1700

Data Index: 5159232 (Open Web Text)

Max Activating Token Index: 217

Text #14

Max Range: 3.5484. Min Range: -3.5484

Max Act: 3.2658. Min Act: -0.1700

Data Index: 2773631 (Open Web Text)

Max Activating Token Index: 33

Text #15

Max Range: 3.5484. Min Range: -3.5484

Max Act: 3.2558. Min Act: -0.1700

Data Index: 6033780 (Open Web Text)

Max Activating Token Index: 930

Text #16

Max Range: 3.5484. Min Range: -3.5484

Max Act: 3.2690. Min Act: -0.1700

Data Index: 5706267 (Open Web Text)

Max Activating Token Index: 878

Text #17

Max Range: 3.5484. Min Range: -3.5484

Max Act: 3.2477. Min Act: -0.1700

Data Index: 1228042 (Open Web Text)

Max Activating Token Index: 707

Text #18

Max Range: 3.5484. Min Range: -3.5484

Max Act: 3.2588. Min Act: -0.1700

Data Index: 6675159 (Open Web Text)

Max Activating Token Index: 756

Text #19

Max Range: 3.5484. Min Range: -3.5484

Max Act: 3.2281. Min Act: -0.1700

Data Index: 6883500 (Open Web Text)

Max Activating Token Index: 332