Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2979 in Layer 3

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.8683. Min Range: -3.8683

Max Act: 3.8116. Min Act: -0.1700

Data Index: 652646 (Open Web Text)

Max Activating Token Index: 244

Text #1

Max Range: 3.8683. Min Range: -3.8683

Max Act: 3.8683. Min Act: -0.1700

Data Index: 8025185 (Open Web Text)

Max Activating Token Index: 829

Text #2

Max Range: 3.8683. Min Range: -3.8683

Max Act: 3.8408. Min Act: -0.1700

Data Index: 8025186 (Open Web Text)

Max Activating Token Index: 790

Text #3

Max Range: 3.8683. Min Range: -3.8683

Max Act: 3.7724. Min Act: -0.1700

Data Index: 317792 (Open Web Text)

Max Activating Token Index: 826

Text #4

Max Range: 3.8683. Min Range: -3.8683

Max Act: 3.7450. Min Act: -0.1700

Data Index: 5203782 (Open Web Text)

Max Activating Token Index: 265

Text #5

Max Range: 3.8683. Min Range: -3.8683

Max Act: 3.7314. Min Act: -0.1700

Data Index: 927953 (Open Web Text)

Max Activating Token Index: 801

Text #6

Max Range: 3.8683. Min Range: -3.8683

Max Act: 3.7277. Min Act: -0.1700

Data Index: 2139191 (Open Web Text)

Max Activating Token Index: 953

Text #7

Max Range: 3.8683. Min Range: -3.8683

Max Act: 3.7063. Min Act: -0.1700

Data Index: 652645 (Open Web Text)

Max Activating Token Index: 304

Text #8

Max Range: 3.8683. Min Range: -3.8683

Max Act: 3.6899. Min Act: -0.1700

Data Index: 497530 (Open Web Text)

Max Activating Token Index: 493

Text #9

Max Range: 3.8683. Min Range: -3.8683

Max Act: 3.6666. Min Act: -0.1700

Data Index: 3145030 (Open Web Text)

Max Activating Token Index: 808

Text #10

Max Range: 3.8683. Min Range: -3.8683

Max Act: 3.6409. Min Act: -0.1700

Data Index: 8661806 (Open Web Text)

Max Activating Token Index: 840

Text #11

Max Range: 3.8683. Min Range: -3.8683

Max Act: 3.6722. Min Act: -0.1700

Data Index: 2139189 (Open Web Text)

Max Activating Token Index: 848

Text #12

Max Range: 3.8683. Min Range: -3.8683

Max Act: 3.6641. Min Act: -0.1700

Data Index: 235883 (Open Web Text)

Max Activating Token Index: 696

Text #13

Max Range: 3.8683. Min Range: -3.8683

Max Act: 3.6363. Min Act: -0.1700

Data Index: 5548299 (Open Web Text)

Max Activating Token Index: 965

Text #14

Max Range: 3.8683. Min Range: -3.8683

Max Act: 3.6188. Min Act: -0.1700

Data Index: 3463992 (Open Web Text)

Max Activating Token Index: 798

Text #15

Max Range: 3.8683. Min Range: -3.8683

Max Act: 3.5891. Min Act: -0.1700

Data Index: 2682857 (Open Web Text)

Max Activating Token Index: 992

Text #16

Max Range: 3.8683. Min Range: -3.8683

Max Act: 3.6444. Min Act: -0.1700

Data Index: 3145029 (Open Web Text)

Max Activating Token Index: 778

Text #17

Max Range: 3.8683. Min Range: -3.8683

Max Act: 3.6021. Min Act: -0.1700

Data Index: 4029451 (Open Web Text)

Max Activating Token Index: 921

Text #18

Max Range: 3.8683. Min Range: -3.8683

Max Act: 3.5681. Min Act: -0.1700

Data Index: 4029457 (Open Web Text)

Max Activating Token Index: 889

Text #19

Max Range: 3.8683. Min Range: -3.8683

Max Act: 3.5896. Min Act: -0.1700

Data Index: 497528 (Open Web Text)

Max Activating Token Index: 607