< Prev | Home | Model | Random | Next >

Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1858 in Layer 7

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('gpt2-small')

Text #0

Max Range: 2.6134. Min Range: -2.6134

Max Act: 2.6134. Min Act: -0.1700

Data Index: 8108248 (Open Web Text)

Max Activating Token Index: 1017

Click toggle to see full text

Truncated

Full Text #0

Text #1

Max Range: 2.6134. Min Range: -2.6134

Max Act: 2.5648. Min Act: -0.1700

Data Index: 4302125 (Open Web Text)

Max Activating Token Index: 1023

Click toggle to see full text

Truncated

Full Text #1

Text #2

Max Range: 2.6134. Min Range: -2.6134

Max Act: 2.5187. Min Act: -0.1700

Data Index: 2375971 (Open Web Text)

Max Activating Token Index: 337

Click toggle to see full text

Truncated

Full Text #2

Text #3

Max Range: 2.6134. Min Range: -2.6134

Max Act: 2.5235. Min Act: -0.1700

Data Index: 3546358 (Open Web Text)

Max Activating Token Index: 477

Click toggle to see full text

Truncated

Full Text #3

Text #4

Max Range: 2.6134. Min Range: -2.6134

Max Act: 2.5240. Min Act: -0.1700

Data Index: 3282519 (Open Web Text)

Max Activating Token Index: 494

Click toggle to see full text

Truncated

Full Text #4

Text #5

Max Range: 2.6134. Min Range: -2.6134

Max Act: 2.5042. Min Act: -0.1700

Data Index: 7828826 (Open Web Text)

Max Activating Token Index: 994

Click toggle to see full text

Truncated

Full Text #5

Text #6

Max Range: 2.6134. Min Range: -2.6134

Max Act: 2.4795. Min Act: -0.1700

Data Index: 1759059 (Open Web Text)

Max Activating Token Index: 874

Click toggle to see full text

Truncated

Full Text #6

Text #7

Max Range: 2.6134. Min Range: -2.6134

Max Act: 2.4929. Min Act: -0.1700

Data Index: 7929388 (Open Web Text)

Max Activating Token Index: 360

Click toggle to see full text

Truncated

Full Text #7

Text #8

Max Range: 2.6134. Min Range: -2.6134

Max Act: 2.4878. Min Act: -0.1700

Data Index: 8667999 (Open Web Text)

Max Activating Token Index: 633

Click toggle to see full text

Truncated

Full Text #8

Text #9

Max Range: 2.6134. Min Range: -2.6134

Max Act: 2.4637. Min Act: -0.1700

Data Index: 4986021 (Open Web Text)

Max Activating Token Index: 690

Click toggle to see full text

Truncated

Full Text #9

Text #10

Max Range: 2.6134. Min Range: -2.6134

Max Act: 2.4649. Min Act: -0.1700

Data Index: 323188 (Open Web Text)

Max Activating Token Index: 745

Click toggle to see full text

Truncated

Full Text #10

Text #11

Max Range: 2.6134. Min Range: -2.6134

Max Act: 2.4744. Min Act: -0.1700

Data Index: 7503941 (Open Web Text)

Max Activating Token Index: 605

Click toggle to see full text

Truncated

Full Text #11

Text #12

Max Range: 2.6134. Min Range: -2.6134

Max Act: 2.4621. Min Act: -0.1700

Data Index: 6994212 (Open Web Text)

Max Activating Token Index: 788

Click toggle to see full text

Truncated

Full Text #12

Text #13

Max Range: 2.6134. Min Range: -2.6134

Max Act: 2.4385. Min Act: -0.1700

Data Index: 1401551 (Open Web Text)

Max Activating Token Index: 824

Click toggle to see full text

Truncated

Full Text #13

Text #14

Max Range: 2.6134. Min Range: -2.6134

Max Act: 2.4519. Min Act: -0.1700

Data Index: 832333 (Open Web Text)

Max Activating Token Index: 713

Click toggle to see full text

Truncated

Full Text #14

Text #15

Max Range: 2.6134. Min Range: -2.6134

Max Act: 2.4558. Min Act: -0.1698

Data Index: 6107939 (Open Web Text)

Max Activating Token Index: 716

Click toggle to see full text

Truncated

Full Text #15

Text #16

Max Range: 2.6134. Min Range: -2.6134

Max Act: 2.4453. Min Act: -0.1700

Data Index: 8119100 (Open Web Text)

Max Activating Token Index: 807

Click toggle to see full text

Truncated

Full Text #16

Text #17

Max Range: 2.6134. Min Range: -2.6134

Max Act: 2.4454. Min Act: -0.1700

Data Index: 1759197 (Open Web Text)

Max Activating Token Index: 364

Click toggle to see full text

Truncated

Full Text #17

Text #18

Max Range: 2.6134. Min Range: -2.6134

Max Act: 2.4397. Min Act: -0.1700

Data Index: 2310013 (Open Web Text)

Max Activating Token Index: 508

Click toggle to see full text

Truncated

Full Text #18

Text #19

Max Range: 2.6134. Min Range: -2.6134

Max Act: 2.4485. Min Act: -0.1700

Data Index: 1872038 (Open Web Text)

Max Activating Token Index: 682

Click toggle to see full text

Truncated

Full Text #19